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This  research  gathers  background  information  to  identify  which  customer 
satisfaction  elements  should  be  included  in  a  standardized  tool  that  measures  the  level  of 
customer  satisfaction  for  AF  Contracting’s  external  and  internal  customers.  A 
comprehensive  literature  review  of  the  prominent  customer  satisfaction  trends  was 
conducted,  while  the  idiosyncrasies  of  customer  satisfaction  that  are  unique  to  AF 
Contracting  were  explored.  For  this  research,  two  customer-specific  questionnaires  were 
used  to  interview  AF  Contracting’s  external  and  internal  customers,  in  order  to  better 
understand  their  experiences.  Based  on  the  results  of  the  interviews,  it  is  apparent  that  AF 
Contracting’s  customers  believe  customer  satisfaction  is  a  critical  component  in  enabling 
effective  communication  and  strengthening  customer  relations. 

Although  AF  Contracting  does  not  currently  use  a  standardized  approach  for 
collecting  customer  satisfaction  infonnation,  this  paper  recommends  the  development  of 
a  customer  satisfaction  mechanism  as  an  essential  tool  to  fully  capitalize  on  the  benefits 
of  improved  communication  and  enhanced  customer  relations.  This  paper  also  proposes  a 
six-step  system  for  developing  a  customer  satisfaction  system  and  specifically  focuses  on 
incorporating  the  customer  satisfaction  elements  as  identified  by  the  customers  who 
participated  in  this  research.  Finally,  this  research  concludes  with  suggestions  for  areas  of 
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I.  INTRODUCTION 


A.  BACKGROUND 

Air  Force  Contracting  is  the  collective  body  of  military  and  civilian  personnel 
who  acquire  capabilities  to  enable  the  global  AF  (Air  Force)  mission  (“Air  Force 
Acquisition,”  2015).  Through  the  effective  implementation  and  management  of  contract 
vehicles,  AF  Contracting  identifies  the  best  suppliers  to  provide  the  best  services  to  the 
nation’s  war  fighters.  There  are  three  main  types  of  contracting:  operational  at  the  wing 
level,  operational  at  the  enterprise-level,  and  systems  (“Air  Force  Installation  &  Mission 
Support  Center,”  2015).  Operational  contracting  at  the  Wing  level  is  focused  on  acquiring 
commodity,  service,  and  construction  capabilities.  These  capabilities  support  day-to-day 
mission  requirements  at  the  Wing  level.  Alternately,  operational  contracting  at  the 
Enterprise  level  seeks  to  strategically  source  commodities  and  services  to  support  the  Air 
Force  enterprise.  Systems  contracting  is  focused  on  weapons  capabilities.  These 
capabilities  support  the  Air  Force’s  needs  of  the  future  while  also  maintaining  its  current 
weaponry  consortium. 

The  Air  Force  Installation  Contracting  Agency  (AFICA)  was  recently  activated  in 
an  effort  to  drive  strategic  sourcing  of  commodities  and  services.  AFICA  now  has 
authority  over  operational  contracting  at  the  Wing  level  and  operational  contracting  at  the 
Enterprise  level.  The  primary  focus  of  AFICA  is  to  assist  in  the  development  and 
execution  of  strategically  sourcing  customer  requirements.  The  Air  Force,  in  its  efforts  to 
leverage  strategic  sourcing,  has  adopted  the  category  management  principle 
(Westermeyer,  2015).  The  category  management  principle  looks  to  categorize  spending 
based  on  the  purchase  of  similar  products  and  services.  AFICA  has  established  a  business 
intelligence  center  which  will  provide  critical  data  analysis  for  AFICA  and  its  customers. 
The  business  intelligence  center  will  gather  and  compare  infonnation  from  AF  databases 
to  industry,  and  create  actionable  data  for  use  by  decision  makers.  The  most  critical 
aspect  of  category  management  is  assigning  a  responsible  owner  to  manage  the  costs  of  a 
particular  category.  This  cost  owner  not  only  manages  costs,  but  also  manages 

consumption  and  coordinates  with  other  uses  of  the  product/service.  Figure  1  depicts 
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category  management  as  it  supports  strategic  sourcing  initiatives.  As  seen  here,  the 
critical  factors  that  support  category  management  are  the  four  pillars:  “Management 
demand  and  consumption,  issue  policy,  strategic/enterprise  solutions,  and  industry  best 
practices”  (Westermeyer,  2015). 


Figure  1 .  The  Four  Pillars  of  Category  Management 


Develop  Business  Intelligence 
Assign  Cost  Ownership  via  Category  Managers 


Categorization  of  Spend 

Source:  R.  Westermeyer,  (2015),  State  of  AFICA:  Current,  and  looking  forward 

[PowerPoint  slides],  Naval  Postgraduate  School. 

Compared  to  the  private  sector,  AF  Contracting  personnel  function  similarly  to 
procurement  or  purchasing  professionals.  Procurement/purchasing  professionals, 
regardless  of  industry,  are  generally  required  to  negotiate,  manage,  and  execute  contract 
actions  to  support  their  organization’s  objectives.  AF  Contracting  personnel  require  the 
same  business  acumen  and  skillsets.  These  skills  are  required  to  properly  engage  and 
cost-effectively  source  the  needs  of  the  Air  Force. 
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An  integral  part  of  sourcing  the  needs  of  the  Air  Force  is  understanding  the 
customer’s  requirements  and  expectations.  The  Air  Force  Contracting  Strategic  Plan 
2009-2013  states,  “Instituting  the  right  business  practices  is  instrumental  in  freeing  our 
resources  to  support  our  customers  and  stakeholders  priorities”  (“Air  Force  Contracting,” 
2015,  p.  3).  Essentially,  to  understand  the  customer  is  to  understand  their  priorities.  To 
better  understand  priorities,  AF  Contracting  needs  a  measurement  tool  for  evaluating 
customer  satisfaction. 

B.  PROBLEM  STATEMENT 

AF  Contracting  does  not  currently  use  a  standardized  approach  for  collecting 
customer  satisfaction  information.  A  standardized  customer  satisfaction  tool  will  enable 
AF  Contracting  to  identify  procedural  inefficiencies,  focus  process  improvement 
initiatives  and  better  serve  its  customers. 

C.  RESEARCH  OBJECTIVES 

The  purpose  of  this  research  is  to  gather  background  information  to  detennine 
what  customer  satisfaction  elements  should  be  included  in  a  standardized  tool,  which 
measures  the  level  of  customer  satisfaction  for  AF  Contracting’s  internal  and  external 
customers.  Through  the  inputs  of  both  customer  bases,  AF  Contracting  will  be  able  to 
comprehensively  understand  its  processes,  procedures,  and  policies  as  viewed  from  the 
perspective  of  its  customers.  The  following  research  questions  are  the  focus  of  this 
project. 

D.  RESEARCH  QUESTIONS 
Primary  Question 

1.  Should  AF  Contracting  develop  a  customer  satisfaction/feedback 
mechanism? 

If  the  answer  to  the  Primary  Question  is  affirmative,  the  Secondary 
Questions  are: 

2.  What  customer  satisfaction  elements  should  we  be  measuring? 
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3.  What  is  the  best  method  for  measuring  customer  satisfaction/feedback? 

4.  Who  is  responsible  for  implementation  and  handling  that  feedback? 

5.  How  can  industry  best  practices  help  with  the  development  of  a  customer 
satisfaction  mechanism? 

E.  METHODOLOGY 

For  gathering  the  data  needed  to  address  the  research  questions,  a  qualitative 
approach  is  used.  The  data  collection  process  consists  of  a  literature  review  and  customer 
interviews.  The  data  will  be  used  to  identify  what  customer  satisfaction  elements  should 
be  included  in  a  standardized  tool  as  part  of  a  customer  satisfaction  system.  A  literature 
review  was  conducted  as  a  basis  for  contextualizing  and  understanding  the  customer 
satisfaction  field.  The  data  collection  process  is  a  combination  of  interviews  from  AF 
Contracting’s  internal  and  external  customers. 

F.  LIMITATIONS 

For  the  scope  of  this  research,  the  customer  satisfaction  measurement  tool  is 
intended  to  support  AF  Contracting  at  the  operational  level.  To  address  customer 
satisfaction  at  the  enterprise  level,  additional  research  would  need  to  be  conducted, 
although  the  findings  in  this  paper  could  be  a  source  for  beginning  that  research.  This 
paper  limits  the  pool  of  participants  due  to  the  extreme  size  of  eligible  AF  Contracting 
customers  across  the  United  States.  However,  the  data  still  provides  generalizable  results 
that  are  reflective  of  the  level  of  satisfaction  that  AF  Contracting’s  customers  experience. 

G.  ASSUMPTIONS 

This  paper  assumes  the  readers  have  a  basic  understanding  of  AF  Contracting’s 
processes:  requirement  definition,  solicitation,  source  selection,  award,  contract 
management  and  contract  close-out.  This  research  assumes  the  sample  size  of  the 
participants  is  large  and  diverse  enough  to  supply  relevant  information  to  the  project. 
Also,  the  sample  size  is  a  fair  representation  of  the  larger  population  of  AF  Contracting’s 
customers. 
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H.  IMPLICATIONS 


A  standardized  customer  satisfaction  tool  will  better  enable  AF  Contracting  to 
identify  procedural  inefficiencies,  focus  process  improvement  initiatives  and  better  serve 
its  customers.  Customer  satisfaction  is  a  critical  component  for  fostering  communication 
and  developing  long-term  relationships.  It  is  essential  for  AF  Contracting  to  employ 
customer  satisfaction  feedback  as  a  strategic  tool  for  improving  operational  effectiveness. 
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II.  LITERATURE  REVIEW 


This  chapter  presents  a  comprehensive  literature  review  on  customer  satisfaction. 
This  literature  review  begins  by  defining  the  customer.  Then,  the  definition  of  customer 
satisfaction  and  how  to  identify  a  satisfied  customer  are  examined.  Next,  the  literature 
review  discusses  the  importance  of  measuring  customer  satisfaction  and  highlights 
federal  regulations  requiring  agencies  to  measure  customer  satisfaction.  Finally,  the 
review  explains  AF  Contracting’s  customer  service  chain. 

A.  WHAT  IS  A  CUSTOMER? 

Prior  to  developing  a  customer  satisfaction  measurement  tool,  it  is  necessary  to 
define  what  and  who  are  “customers.”  Customers  are  typically  viewed  as  either  internal 
or  external  and  identified  in  this  manner  according  to  the  scope  of  the  organization 
(United  States  Office  of  Personnel  Management,  1997).  In  the  early  2000s,  the  Total 
Quality  Management  (TQM)  model  gained  traction  in  the  business  world  for  its 
“customer  focus”  component  of  quality  management  (Sirvanci,  2004).  One  popular  TQM 
definition  is,  “a  continuously  evolving  management  system  consisting  of  core  values, 
methodologies  and  tools,  the  aim  of  which  is  to  increase  external  and  internal  customer 
satisfaction  with  a  reduced  amount  of  resources”  (Klefsjo,  Bergquist,  &  Garvare,  2008,  p. 
121).  Based  on  this  working  definition,  TQM  acknowledges  the  distinction  between 
internal  and  external  customers,  both  of  which  should  be  evaluated  for  their  level  of 
satisfaction. 

Klefsjo  et  al.  (2008),  discuss  three  definitions  of  a  customer  spanning  from  a  very 
narrow  definition  to  a  very  broad  one.  According  to  the  authors,  the  narrower  definitions 
are  developed  by  the  International  Organization  for  Standardization  (ISO)  90001:2000, 
one  of  which  is:  “...internal  or  external,  an  organization  that,  or  a  person  who  receives  a 
product,”  or  alternately,  “actual  and  potential  users  of  your  organizations  products  and 
services,”  as  suggested  by  the  National  Institute  for  Standards  and  Technology  (p.  124). 
These  definitions  limit  the  scope  of  who  qualifies  as  a  customer,  particularly  excluding 
suppliers  as  customers.  The  authors  offer  the  intermediary  definition  of  a  customer, 
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according  to  Garvare  and  Johansson,  2007,  as:  “Customers  as  individuals  or 
organizations  being  down-stream  in  the  product  life  cycle  process,  that  is,  receivers  of  a 
product”  (p.  124).  This  latter  definition  is  slightly  broader  than  the  previous  one  and 
flexes  to  include  suppliers  as  customers. 

The  broadest  definitions  are  practically  inclusive  of  anyone,  making  it  difficult  to 
consistently  apply  those  definitions  across  multiple  industries  and  organizations.  The 
authors  credit  Bergman  and  Klefsjo  (2003),  with  the  definition  of  a  customer  as  “those 
we  want  to  create  value  for”  (p.  124).  Similarly,  Juran’s  Quality  Control  Handbook, 
defines  a  customer  as:  “Anyone  who  is  impacted  by  the  product  or  by  the  process  used  to 
produce  the  product”  (Juran  &  Gryna,  1988,  p.  6.4).  These  overly  broad  definitions 
frequently  lead  to  miscommunication  and  misunderstanding  about  who  should  be 
included  in  customer-focused  initiatives  (Klefsjo  et  ah,  2008,  p.  124).  Considering  all  the 
different  definitions  of  a  customer,  it  is  very  important  for  an  organization  to  select  a 
definition  that  is  truly  inclusive  of  all  their  pertinent  customers. 

For  the  purposes  of  this  study,  the  following  definition  of  a  customer  is  used: 
“Anyone  who  is  affected  by  the  product  or  by  the  process  used  to  produce  the  product” 
(Klefsjo  et  ah,  2008,  p.  124).  This  definition,  although  broad,  considers  both  external  and 
internal  customers.  External  customers  are  “affected  by  the  product,”  whereas  internal 
customers  are  “affected  by  the  process  used  to  produce  the  product.”  From  a  Supply 
Chain  Management  (SCM)  perspective,  an  external  customer  is  a  “downstream 
customer”  and  an  internal  customer  is  an  “upstream  supplier”  (Ou,  Liu,  Hung  &  Yen, 
2010,  p.  527).  When  fluid  communication  up  and  downstream  occurs,  improved 
customer  satisfaction  is  inevitable  (Ou  et  ah,  2010). 

B.  WHO  ARE  AF  CONTRACTING’S  CUSTOMERS? 

Based  on  the  previously-established  definition  of  a  customer,  AF  Contracting 
serves  both  internal  and  external  customers.  The  external  customers  develop  the 
requirements  needing  outsourcing  and  bring  those  requirements  to  AF  Contracting 
personnel  for  execution.  The  Air  Force  is  composed  of  the  following  10  Major 
Commands  (MAJCOMs):  Air  Combat  Command  (ACC);  Air  Force  Space  Command 
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(AFSPC);  Air  Mobility  Command  (AMC);  Air  Force  Material  Command  (AFMC) 
United  States  Air  Forces  Europe  (USAFE);  Pacific  Air  Forces  (PACAF);  Air  Force 
Global  Strike  Command  (AFGSC);  Air  Force  Special  Operations  Command  (AFSOC); 
Air  Education  and  Training  Command  (AETC)  and  Air  Force  Reserve  Command 
(Westermeyer,  2015;  Air  Force  Historical  Research  Agency,  2015).  The  commands  are 
responsible  for  executing  a  specific  mission  and  requiring  a  unique  portfolio  of  goods  and 
services.  AF  Contracting  is  solely  responsible  for  supporting  each  command’s 
requirements. 

Effective  May  5,  2015,  the  AF  activated  a  new  organization  to  centrally  manage 
“installation  and  mission  support  capabilities,”  which  allows  “the  Air  Force  to  resource 
and  sustain  a  standard  level  of  support  based  on  Air  Force  priorities”  (Air  Force 
Installation  and  Mission  Support  Center,  2015).  This  new  organization,  Air  Force 
Installation  and  Mission  Support  Center  (AFIMSC),  combines  the  efforts  of  “ten  major 
commands,  two  direct  reporting  units  and  multiple  field  operating  agencies”  (AFIMSC, 
2015).  AFIMSC  aligns  the  following  six  mission  support  functions  under  the  same 
command  structure:  Air  Force  Civil  Engineer  Center  (AFCEC);  Air  Force  Financial 
Services  Center  (AFFSC);  Air  Force  Installation  Contract  Agency  (AFICA);  Air  Force 
Security  Forces  Center  (AFSFC);  Air  Force  Financial  Management  Center  of  Expertise 
and  Air  Force  Services  Activity  (AFSVA)  (AFIMSC,  2015).  With  the  activation  of  this 
organization,  AF  Contracting  is  now  headquartered  with  several  of  its  primary  external 
customers.  According  to  Westermeyer  (2015),  “AFIMSC  affords  unprecedented 
opportunity  to  team  across  the  Center,”  and  is  perfectly  structured  for  obtaining  customer 
satisfaction  feedback  at  the  senior  leadership  level  (12). 

AF  Contracting’s  internal  customers  are  the  collective  group  of  suppliers  who 
provide  goods  and  services  to  the  external  customer.  Across  the  globe,  there  are 
thousands  of  suppliers  actively  supporting  AF  Contracting  via  a  customized  contracting 
vehicle.  AF  Contracting’s  internal  customers  range  from  very  large  corporations  like 
Boeing,  Lockheed  Martin,  and  General  Electric,  to  small  businesses  such  as  Apexio 
Solutions  Inc.,  Earthwinds  LLC,  and  Pelatron  Inc.  There  is  currently  no  centralized  body 
of  suppliers;  rather,  every  AF  Contracting  unit  employs  a  portfolio  of  suppliers  to  support 
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their  location.  AF  Contracting,  through  the  governance  of  AFICA,  intends  to  optimize  its 
base  of  suppliers  through  strategic  sourcing  solutions  (Landale,  2015). 


C.  WHAT  IS  CUSTOMER  SATISFACTION? 

Richard  Oliver,  a  pioneer  in  customer  satisfaction  research,  is  quoted  as  saying, 
“Everyone  knows  what  satisfaction  is  until  asked  to  give  a  definition.  Then  it  seems, 
nobody  knows”  (Zeithaml,  Bitner,  &  Gremler,  2013,  p.  80).  Without  a  clear 
understanding  of  what  “customer  satisfaction”  is,  it  becomes  increasingly  difficult  to 
measure.  Customer  satisfaction  is  a  broad  term,  loosely  used  by  servicing  organizations 
to  evaluate  a  customer’s  experience.  Correspondingly,  organizations  often  use  customer 
satisfaction  as  an  indicator  of  the  organization’s  service  or  product  perfonnance 
(Anderson  &  Sullivan,  1993).  The  tenn  customer  satisfaction  is  typically  determined  by 
the  customer’s  expectations,  perfonnance,  and  disconfirmation;  collectively,  these  factors 
describe  the  Expectancy  Disconfinnation  Paradigm  (EDP)  (Churchill  &  Surprenant, 
1982). 

Figure  2  models  the  EDP  logic,  showing  the  interconnectivity  of  each  factor.  The 
“CS”  at  the  end  of  the  model  represents  customer  satisfaction.  For  the  purpose  of  this 
study,  perceived  perfonnance  and  perfonnance  are  used  interchangeably  because  they 
reference  the  same  aspect  of  customer  satisfaction.  The  question  marks  signify  the  mixed 
findings  of  several  studies  where  expectations  may  directly  affect  customer  satisfaction, 
with  little  influence  from  perfonnance.  Similarly,  performance  may  directly  affect 
customer  satisfaction,  with  little  influence  from  expectations  (Yi,  1993).  These  findings 
highlight  the  limitations  of  this  model,  but  AF  Contracting  can  still  glean  valuable 
insights  about  its  customers’  behaviors  and  how  AF  Contracting  can  influence  their 
satisfaction. 
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Figure  2.  Expectation  of  Discontinuation  Model 


Source:  Y.  Yi,  (1993),  The  determinants  of  consumer  satisfaction:  The  moderating  role  of 
ambiguity,  Advances  In  Consumer  Research,  20,  502-506. 


EDP  is  a  customer  satisfaction  theory  from  the  1970s.  The  theory  correlates  the 
effects  of  a  customer’s  preconceived  expectations  with  the  performance  of  a  product  or 
service.  These  preconceived  expectations  either  become  validated  or  disproven,  and  the 
result  is  positive  or  negative  disconfirmation.  If  the  disconfirmation  is  positive,  a 
customer  is  considered  satisfied.  Alternately,  a  negative  disconfirmation  is  synonymous 
with  being  dissatisfied  (Pallister,  Rosidah  &  Robson,  2015).  With  this  foundational 
understanding,  organizations  can  tailor  their  efforts  to  maximize  a  customer’s 
satisfaction. 

According  to  EDP,  a  customer’s  preconceived  expectation  is  their  anticipatory 

response  to  how  a  product  or  service  should  perform.  Outside  the  scope  of  this  study  but 

equally  valuable,  are  several  environmental,  social  and  behavioral  factors  that  influence  a 

customer’s  preconception  (Churchill  &  Surprenant,  1982).  Internal  and  external 

customers  formulate  their  expectations  about  how  AF  Contracting  should  operate  based 

on  their  knowledge,  experiences  with  and  understanding  about  the  contracting  process. 

AF  Contracting  can  use  this  knowledge  to  influence  their  customers’  preconceptions 
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through  early  engagement  and  education  about  the  purchasing  process.  Equally 
important,  the  availability,  communication,  professional  knowledge  and  responsiveness 
of  AF  Contracting  personnel  can  positively  influence  a  customer’s  preconceptions 
(Montgomery  County  Government,  2012). 

Pallister  et  al.  (2015)  echo  Czepiel’s  (1990)  definition  of  perfonnance  as  “the 
customer’s  evaluation  of  product  or  service  perfonnance  following  the  consumption 
experience”  (p.  350).  Essentially,  once  the  customer  receives  their  product  or  service,  the 
evaluation  for  acceptability  begins.  Pallister  et  al.  (2015),  further  expand  their  definition 
to,  “subjective  evaluations  of  the  core  product  (i.e.,  attributes  of  the  focal  product), 
comprising  both  intrinsic  (effectiveness)  and  extrinsic  (packaging)  characteristics”  (p. 
350).  This  is  to  say,  acceptability  of  the  product  or  service  includes  attributes  such  as: 
delivery  time,  condition  of  the  delivery,  effectiveness  of  contracted  services  and 
professionalism  of  contracted  service  providers.  AF  Contracting  can  positively  influence 
the  perceived  performance  of  its  external  customers  when  relationships  with  their  internal 
customers  are  strong  and  the  product/service  is  acceptable.  Also,  internal  customers  can 
be  positively  influenced  when  AF  Contracting  clearly  articulates  and  defines  the  external 
customer’s  requested  good  or  service. 

Disconfirmation  is  the  summation  of  the  customer’s  expectations  plus  perceived 
performance.  The  level  of  disconfirmation  distinguishes  satisfaction  from  dissatisfaction 
(Churchill  &  Surprenant,  1982,  p.  493).  When  a  customer’s  expectations  exceed 
performance,  dissatisfaction  is  likely.  Conversely,  when  a  customer’s  expectations  are 
superseded  by  performance,  satisfaction  is  likely.  Managing  expectations  and 
performance  is  necessary  to  positively  influence  customer  satisfaction.  AF  Contracting 
must  continually  monitor  its  internal  and  external  customers’  level  of  disconfirmation 
with  the  intent  to  positively  influence  customer  satisfaction. 

Churchill  &  Surprenant  (1982)  discuss  an  aspect  of  satisfaction  worthy  of 

consideration:  “Satisfaction  is  similar  to  attitude  in  that  it  can  be  assessed  as  the  sum  of 

the  satisfactions  with  the  various  attributes  of  the  product  of  service”  (p.  493).  Customers 

can  be  satisfied  with  the  product  but  not  satisfied  with  the  customer  service  they  received. 

Similarly,  a  customer  can  be  satisfied  with  the  customer  service  and  dissatisfied  with  the 
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product.  In  either  situation,  the  probability  of  a  dissatisfied  customer  is  high.  This  is  an 
aspect  of  satisfaction  that  AF  Contracting  must  be  mindful  of  when  engaging  with  and 
serving  its  customers. 

D.  VALUE  OF  MEASURING  CUSTOMER  SATISFACTION  IN  AF 

CONTRACTING 

Lord  Kelvin  (1907,  originally  published  in  1824)  stated:  “when  you  can  measure 
what  you  are  speaking  about,  and  express  it  in  numbers,  you  know  something  about 
it...  [otherwise]  your  knowledge  is  of  a  meagre  and  unsatisfactory  kind  “(Neely,  Gregory 
&  Platts,  2005,  p.  1228).  Measuring  customer  satisfaction  is  important  because  it 
indicates  an  organization’s  awareness  and  understanding  of  what  is  valuable  to  the 
customer.  Identifying  what  is  valuable  to  the  customer  gives  the  organization  a 
competitive  advantage  (Asher,  1989).  An  additional  benefit,  as  stated  by  Anderson, 
Fornell,  and  Lehmann  (1994)  is,  “firms  that  achieve  high  customer  satisfaction  also  enjoy 
superior  economic  returns”  (p.  63).  Organizations  are  constantly  lobbying  to  attract  and 
retain  customers  (Kendall,  2008),  and:  “The  fundamental  reason  customer  satisfaction  is 
important  to  your  organization  is  because  it  allows  your  organization  to  stay  in  business” 
(Deviney,  1998,  p.  16).  When  an  organization  invests  in  customer  satisfaction,  it  typically 
yields  increased  customer  loyalty,  higher  profit  margins  and  subsequently  increases 
performance  (Anderson,  Pearo,  &  Widener,  2008). 

Although  AF  Contracting  is  not  a  profit-driven  organization,  superior  customer 
satisfaction  will  strengthen  the  efficiency  and  effectiveness  of  the  entire  service  chain 
(Neely  et  ah,  2005).  Increases  in  efficiency  and  effectiveness  result  in  decreases  in  waste 
and  ultimately  decreases  in  costs.  This  decrease  in  costs  enables  AF  Contracting’s 
external  customers  to  acquire  more  mission  capability.  Mission  capability  is  often  a 
function  of  the  number  of  projects  a  customer  completes  within  their  budgetary 
constraints.  Figure  3  provides  a  visual  example  of  an  external  customers’  increased 
mission  capability.  External  customers  create  Priority  Lists  (PLs)  that  reflect  their 
mission  needs.  For  example,  Air  Force  Security  Forces  Center  (AFSFC)  may  compile  a 
list  of  mission  capabilities  like  veterinary  services  for  their  military  working  dogs,  new 

protective  gear  or  software  upgrades  for  the  base  security  system.  AFSFC  will  then  rank 
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the  capabilities  by  priority.  As  seen  in  Figure  3,  PL  #1  is  a  sample  list  of  20  projects 
where  10  projects  are  able  to  be  completed  within  budget.  PL  #2  is  the  same  list  of 
projects  but  15  of  20  projects  are  able  to  be  completed  without  increasing  the  budget. 
Delivering  more  capability  to  the  customer  is  the  objective  for  AF  Contracting  and 
measuring  customer  satisfaction  assists  in  meeting  this  objective  (Westermeyer,  2015). 


Figure  3.  External  Customer  Project  List  Comparison 


Source:  R.  Westermeyer,  (2015),  State  of  AFICA:  Current,  and  looking  forward 
[PowerPoint  slides],  Naval  Postgraduate  School. 


Ultimately,  measuring  customer  satisfaction  is  important  because  it  influences 
actions  and  affects  decisions.  Short-term  and  long-term  decisions  will  both  be  affected  by 
how  customer  satisfaction  is  measured  (Hauser  &  Katz,  1998).  If  organizations  know 
their  external  customers  are  unhappy  with  the  service  or  product  they  have  received,  the 
organization  will  have  to  decide  how  to  react.  If  an  internal  customer  is  consistently 
underperforming,  the  organization  will  also  decide  how  to  react.  Whether  making 
strategic  level  decisions  affecting  the  enterprise  or  operational  decisions  affecting  a  local 
unit,  customer  satisfaction  will  affect  decision  making,  therefore;  it  is  imperative  to 
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measure  customer  satisfaction.  In  a  fiscally  constrained  environment  with  limited 
resources,  every  decision  warrants  the  input  of  the  customer. 

E.  IMPLEMENTING  CUSTOMER  SATISFACTION:  METHODS  AND 

FREQUENCY 

Ways  to  measure  customer  satisfaction  have  been  extensively  researched,  and 
according  to  Wilson  (2002),  usually  consumes  the  largest  portion  of  a  firm’s  market 
research  budget  (Olsen,  Witell  &  Gustafsson,  2014).  With  all  the  research  and  popularity 
surrounding  Customer  Relationship  Management  (CRM)  and  customer  satisfaction,  it  is 
surprising  to  find  “studies  on  how  service  firms  collect  and  implement  relevant  customer 
satisfaction  infonnation  are  scarce”(Morgan  et  ah,  2005  &  Olsen  et  ah,  2014).  CRM  has 
dominated  business  strategy  for  many  years  but  recent  literature  is  finding  “that  the 
contemporary  consumer  seeks  more  than  competent  services  but  also  experiences....” 
(Nasution,  Sembada,  Miliani,  Resti  &  Prawono,  2014,  p.  255).  The  emerging  concept  of 
Customer  Experience  Management  (CEM)  has  been  broadly  defined  by  several  scholars, 
but  Nasution  et  al.  (2014)  refer  to  Swinyard’s  (1993)  CEM  definition  of,  “service 
perceptions  throughout  each  touchpoint  with  the  firm”  (p.  255).  CEM  provides  deeper 
insights  into  how  an  organization  can  implement  customer  satisfaction. 

CEM  proponents  Meyer  &  Schwager  (2007)  argue  that  the,  method  and  collection 
frequency  of  customer  satisfaction  data  are  detennined  by  the  specific  information  the 
organization  wants  to  evaluate.  Past,  Present  and  Potential  patterns  are  the  specific 
categories  of  CEM  that  detennine  the  method  for  collecting  and  analyzing  data.  Each 
category  produces  different  infonnation  and  understanding  of  the  customer.  An 
organization  choosing  to  analyze  a  customer’s  past  pattern  will  collect  data  and  employ 
analysis  methods  differently  than  an  organization  choosing  to  analyze  present  or  potential 
patterns.  Organizations  are  free  to  blend  the  categories  as  needed  to  achieve  the  insights 
they  desire. 

According  to  Meyer  &  Schwager  (2007),  an  organization  will  analyze  Past 
Patterns  to  collect  recent  customer  experiences.  The  primary  goals  are:  tracking  trends, 
analyzing  new  initiatives  and  evaluating  the  success  or  failure  of  a  new  product  or 
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initiative.  Present  Patterns  capture  current  customer  experiences  and  forecast  potential 
opportunities.  The  primary  goal  is  monitoring  and  strengthen  customer  relationships.  The 
last  category  of  CEM  is,  Potential  Patterns  which  focus  exclusively  on  future 
opportunities. 

Let  us  consider  how  a  Present  Pattern  analysis  might  look.  Present  Pattern 
analysis  recommends  collecting  customer  satisfaction  quarterly  and  holds  the  business 
units  accountable  for  managing  the  data.  According  to  Meyer  and  Schwager  (2007),  the 
recommended  method  for  collecting  customer  satisfaction  data  is  through  “web-based 
surveys  preceded  by  preparation  in  person,  direct  contact  in-person  or  by  phone,  user 
forums,  focus  groups  [or  any  other]  regularly  scheduled  formats”  (p.  6).  The  benefit  of  an 
Internet-based  method  is  the  rapid  flow  of  infonnation.  Immediately  after  customers 
provide  a  response,  the  infonnation  is  accessible.  With  the  proper  software,  the 
information  is  sorted  and  analyzed  with  the  click  of  a  button.  The  Present  Pattern  also 
encourages  analysis  at  the  “corporate,  business  unit  or  local  level”  with  major  trend 
issues  being  elevated  to  the  “general  manager”  level  (p.  6). 

The  optimal  time  to  implement  the  recommendations  from  the  Past,  Present  and 
Potential  Patterns  is  during  their  initial  implementation.  Once  the  system  is  established 
and  a  steady  stream  of  communication  is  flowing,  the  frequency  of  data  collection  can 
decrease  to  semi-annually,  annually  or  as  deemed  appropriate  by  the  organization.  The 
data  collection  methods  can  also  be  periodically  reviewed  to  keep  pace  with 
technological  improvements. 

F.  CUSTOMER  SERVICE  FOCUS  IN  THE  FEDERAL  GOVERNMENT 

1.  Executive  Orders 

Customer-focused  service  has  been  a  federal  government  priority  since  the 
issuance  of  Executive  Order  No.  12862,  Setting  Customer  Service  Standards.  On 
September  1 1th,  1993,  President  Clinton  issued  the  order  to  federal  government  agencies 
as  an  initiative  to  ensure  American  citizens  receive  premier  and  quality  service.  The 
purpose  of  the  order  was  to  establish  a  set  of  customer  service  standards  applicable  across 
the  entire  federal  government  (Executive  Order  No.  12862,  1993).  Ultimately  the 
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objective  is  “for  Federal  agencies  to  deliver  customer  service  that  equals  the  best  in 
business”  (United  States  Office  of  Personnel  Management,  1997,  p.  1).  The  value  of 
instituting  customer  service  standards  is  not  only  to  provide  American  citizens  with  great 
service  but  is  also  to  create  “a  government  that  works  better  and  costs  less”  (Wellens  & 
Martin,  1995,  p.  1).  President  Clinton  published  the  following  customer  service 
standards: 

(a)  identify  the  customers  who  are,  or  should  be,  served  by  the  agency; 

(b)  survey  customers  to  detennine  the  kind  and  quality  of  services  they 
want  and  their  level  of  satisfaction  with  existing  services; 

(c)  post  service  standards  and  measure  results  against  them; 

(d)  benchmark  customer  service  performance  against  the  best  in  business; 

(e)  survey  front-line  employees  on  barriers  to,  and  ideas  for,  matching  the 
best  in  business; 

(f)  provide  customers  with  choices  in  both  the  sources  of  service  and  the 
means  of  delivery; 

(g)  make  information,  services  and  complain  systems  easily  accessible; 
and 

(h)  provide  means  to  address  customer  complaints.  (Executive  Order  No. 
12862,2011) 

On  April  27,  2011,  President  Obama  issued  Executive  Order  13571,  Streamlining 
Service  Delivery  and  Improving  Customer  Service.  This  order  stems  from  Executive 
Order  12862,  and  states  “the  public  deserves  competent,  efficient  and  responsive  service 
from  the  Federal  Government”  (Executive  Order  No.  13571,  2011).  In  the  18-year  gap 
between  Executive  Orders,  the  public  has  increased  their  expectation  of  transparency 
from  the  Federal  Government.  Technological  advances  over  the  past  18-years  have 
provided  more  efficient  service  delivery  systems  like  “lower-cost,  self-service  options 
accessed  by  the  Internet  or  mobile  phone  and  improved  processes  that  deliver  services 
fast  and  more  responsively.  The  Federal  Government  has  a  responsibility  to  streamline 
and  make  more  efficient  its  service  delivery  to  better  serve  the  public”  (Executive  Order 
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No.  13571,  2011).  Each  agency  was  charged  with  developing  a  Customer  Service  Plan. 
The  plans  must  address  six  key  actions  for  agencies  to  implement: 

(a)  establishing  one  major  initiative  (signature  initiative)  that  will  use 
technology  to  improve  the  customer  experience; 

(b)  establishing  mechanisms  to  solicit  customer  feedback  on  Government 
services  and  using  such  feedback  regularly  to  make  service  improvements; 

(c)  setting  clear  customer  service  standards  and  expectations,  including, 
where  appropriate,  performance  goals  for  customer  service  required  by  the 
GPRA  (Government  Perfonnance  and  Results)  Modernization  Act  of  2010 
(Public  Law  1 1 1-352); 

(d)  improving  the  customer  experience  by  adopting  proven  customer 
service  best  practices  and  coordinating  across  service  channels  (such  as 
online,  phone,  in-person,  and  mail  services); 

(e)  streamlining  agency  processes  to  reduce  costs  and  accelerate  delivery, 
while  reducing  the  need  for  customer  calls  and  inquiries;  and 

(f)  identifying  ways  to  use  innovative  technologies  to  accomplish  the 
customer  service  activities  above,  thereby  lowering  costs,  decreasing 
service  delivery  times,  and  improving  the  customer  experience.  (Executive 
Order  No.  13571,2011) 

Beginning  with  the  President,  the  Federal  Government  understands  the  value  of 
investing  in  customer  satisfaction  initiatives.  Seizing  the  opportunity  to  become  more 
efficient  with  taxpayer  dollars  and  reduce  unnecessary  processes  is  an  important  step 
towards  achieving  the  president’s  vision  of  improving  “the  customer  experience” 
(Executive  Order  No.  13571,  2011).  In  light  of  a  shrinking  fiscal  budget,  the  efforts  to 
mirror  private  sector  practices  may  lead  to  reduced  cost  and  better  service. 

2.  ICE  in  the  DOD 

While  the  executive  government  focuses  on  creating  orders  and  providing 
guidance  to  federal  agencies,  the  Department  of  Defense  (DOD)  implemented  a  customer 
satisfaction  feedback  tool  for  service-focused  agencies.  The  customer  satisfaction 
feedback  tool  for  the  DOD  is  known  as  the  Interactive  Customer  Evaluation  (ICE) 
system.  The  push  for  the  ICE  system  began  in  1998  when  the  European  Command 
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(EUCOM)  Deputy  Commander  requested  a  customer  evaluation  system  that  was  efficient 
and  provided  timely  data  on  a  customer’s  experience.  As  a  result,  the  initial  ICE  system 
was  created.  A  few  years  later,  in  2001,  the  ICE  version  2  was  deployed  as  a  more  robust 
and  globalized  system  that  is  still  being  used  today  (ICE  Administrator,  personal 
communication,  2015). 

Currently,  the  ICE  system  is  a  web-based  customer  satisfaction  feedback  tool 
used  by  DOD  entities  across  the  globe.  The  system  allows  customers  to  provide 
anonymous  feedback  about  their  interaction  with  an  organization  through  a  comment 
card.  The  comment  card  has  6  standard  questions,  5  of  which  can  be  removed  as  the 
organization  deems  appropriate.  The  ICE  program  recommends  limiting  the  total  number 
of  questions  to  no  more  than  10,  but  the  questions  are  tailored  to  meet  the  organization’s 
specific  needs.  Comment  cards  are  a  common  practice  and,  when  combined  with  the 
speed  and  flexibility  of  the  Internet,  they  become  a  valuable  tool  for  the  Commander  in 
assessing  a  customer’s  level  of  satisfaction  (ICE  Administrator,  personal  communication, 
2015). 

Utilization  of  the  ICE  system  spans  across  all  branches  of  the  military  and  several 
OSD  agencies,  including  Defense  Finance  and  Accounting  Service  (DFAS)  and 
Washington  Headquarters  Services  (WHS).  Over  540  ICE  sites,  24,000  comment  cards 
created  by  feedback  seekers  and  2  million  comment  cards  that  have  been  submitted  by 
customers  (ICE  Administrator,  personal  communication,  2015).  The  ICE  system  is 
clearly  being  used  by  customers,  but  it  is  not  consistently  used  across  AF  Contracting 
units. 

An  example  of  an  AF  Contracting  unit  that  has  created  a  comment  card  on  the 
ICE  system  is  located  at  Cannon  AFB.  Their  comment  card  is  designed  to  obtain 
feedback  from  resource  advisors,  end  users,  technical  experts,  contracting  officer 
representatives  (CORs),  and  contractors.  All  of  these  positions  represent  either  internal  or 
external  customers  throughout  the  contracting  process  and  therefore  can  provide  valuable 
feedback  .  Although  the  site  does  not  show  how  frequently  the  comment  cards  are  used  or 
how  effective  the  data  is  to  the  commander,  the  ICE  system  does  provide  a  free  template 

on  the  site  to  use  for  developing  a  customer  satisfaction  tool. 
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G.  AF  CONTRACTING’S  INTEGRATED  SUPPLY  CHAIN 

A  supply  chain  is  the  aligning  of:  “three  or  more  organizations  linked  directly  by 
one  or  more  upstream  or  down  stream  flows  of  products,  services,  finances  and 
information  from  a  source  to  a  customer”  (Hawkins,  2015).  Supply  chains’  growth  to 
multiple  levels  is  possible  depending  upon  the  intricacies  of  the  organizations  within  the 
chain.  Supply  chains  become  Supply  Chain  Management  (SCM)  when  “the  planning  and 
management  of  all  activities  involved  in  sourcing  and  procurement,  conversion  and  all 
logistics  management  activities,”  are  combined  (Hawkins,  2015).  SCM  has  continuously 
evolved  over  the  years.  Many  businesses  use  SCM  as  a  barometer  for  organizational 
performance  and  to  gain  a  competitive  advantage  over  other  vendors  (Deshpande,  2012). 
Integrated  supply  chain  is  an  extension  of  SCM.  Each  organization  in  the  supply  chain 
plans  their  activities  in  coordination  with  other  relevant  organizations,  and  requires  a 
streamlined  flow  of  information  (Lambert  &  Cooper,  2000  &  Fox,  Chionglo  & 
Barbuceanu,1993). 

Figure  4  provides  an  example  of  an  integrated  supply  chain,  representing  the 
relationships  between  AF  Contracting,  its  external  customers,  and  its  internal  customers. 
This  integrated  supply  accounts  for  AF  Contracting’s  external  and  internal  customers. 
Cainm  (2002)  developed  an  integrated  supply  chain  graphic,  entitled  “Players  Relevant  to 
an  Integrated  Air  Force  Supply  Chain  That  Includes  Contractors”  (p.402).  This  graphic 
shows  several  of  the  important  players  in  AF  Contracting’s  supply  chain.  Many  of  the 
players  are  similarly  reflected  in  Figure  4.  AF  Contracting’s  internal  customers  consist  of 
prime  contractors  and  subcontractors.  Prime  contractors  are  AF  Contracting’s  main 
suppliers  of  goods  and  services,  whereas  subcontractors  are  the  main  suppliers  to  AF 
Contracting’s  prime  contractors.  The  reason  for  the  distinction  between  a  prime 
contractor  and  a  subcontractor  is  because  AF  Contracting  only  negotiates  and  forms  a 
contract  with  a  prime  contractor.  The  interaction  with  a  subcontractor  is  very  limited, 
although  the  subcontractor  is  a  vital  member  of  the  supply  chain.  The  double  arrows 
between  AF  Contracting  and  the  prime  contractors,  and  the  prime  contractors  and  their 
subcontractors,  show  the  relationship  and  communication  exchange  that  currently  exists. 
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Figure  4.  AF  Contracting  Integrated  Supply  Chain 


Internal  Customers 


External  Customers 


The  external  customers’  side  of  the  supply  chain  shows  the  functional  owners 
who  are  requesting  a  good  or  service  from  AF  Contracting.  As  described  by  Camm 
(2002),  a  functional  “is  shorthand  for  an  organization  with  a  specific  functional 
responsibility  for  the  career  field  (‘community’)  of  personnel  who  maintain  skills 
required  to  execute  specialized  activities  associated  with  a  functional  responsibility”  (p. 
402).  Examples  of  a  functional  owner  include:  Civil  Engineering,  Aircraft  Maintenance, 
and  Medical.  AF  Contracting  serves  multiple  functionals.  Functional  representatives 
request  goods  and  services  to  preserve  and  increase  their  mission  capabilities.  It  is 
mission  capabilities  that  enable  the  warfighter  to  execute  their  tasks  in  defense  of  the 
country. 

The  integrated  supply  chain  is  a  valuable  tool  and  helps  convey  the  essential  role 
of  each  organization.  This  chain  also  shows  the  importance  of  establishing  clear  channels 
of  communication  with  both  the  internal  and  external  customers.  If  the  AF  establishes  a 
well-functioning  chain  with  satisfied  customers,  then  they  are  better  positioned  to  achieve 
their  goals. 
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H.  SUMMARY 


This  chapter  provides  a  comprehensive  overview  of  literature  relevant  to  customer 
satisfaction.  This  literature  review  first  defines  a  customer  as  it  pertains  to  AF 
Contracting.  It  then  identifies  who  AF  Contracting  customers  are  and  distinguishes 
between  internal  and  external  customers.  Next,  it  discusses  what  customer  satisfaction  is 
through  the  use  of  the  EDP  model,  and  then  assesses  the  value  and  importance  of 
measuring  customer  satisfaction  through  the  lens  of  AF  Contracting.  The  value  of 
measuring  customer  satisfaction  equates  to  AF  Contracting’s  ability  to  deliver  more 
capability  within  the  same  budgetary  limitations.  Literature  pertaining  to  implementing 
customer  satisfaction  is  also  reviewed.  Although  there  are  many  factors  to  consider  when 
implementing  customer  satisfaction,  the  method  and  frequency  of  implementation  are  the 
focus.  The  chapter  then  discusses  customer  service  regulations  for  the  federal  government 
in  particular  the  two  key  statutes  that  focus  on  improving  customer  service.  Lastly,  this 
chapter  illustrates  AF  Contracting’s  integrated  supply  chain.  The  purpose  of  the  chain  is 
to  highlight  the  internal  and  external  customers’  relationships  with  AF  Contracting. 
Altogether,  this  chapter  establishes  a  foundation  that  supports  the  need  for  AF 
Contracting  to  develop  a  customer  feedback  mechanism  to  evaluate  both  internal  and 
external  customers. 
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III.  METHODOLOGY 


This  chapter  describes  the  methodology  employed  to  conduct  this  research.  The 
chapter  begins  with  a  discussion  about  the  sample  selected.  Next,  the  chapter  will 
describe  the  subject  selection  process  followed  by  an  explanation  of  the  development  of 
the  interview  questionnaire.  Following  the  interview  questionnaire  discussion,  the  data 
collection  process  is  explained  and  the  procedure  used  to  analyze  the  data  once  the 
interviews  were  complete  is  provided. 

A.  SAMPLE 

This  research  uses  a  representative  sample  of  AF  Contracting’s  internal  and 
external  customers  in  order  to  equitably  characterize  the  type  of  customers  AF 
Contracting  serves.  It  would  have  been  unfeasible  to  interview  all  of  AF  Contracting’s 
internal  and  external  customers,  therefore  a  smaller  representative  sample  was  chosen. 
The  sample  size  was  approximated,  based  on  the  scope  of  the  study,  and  is  limited  to 
local  area  participants.  Local  area  participants  include  internal  and  external  customers 
who  support  Travis  Air  Force  Base  (AFB)  and  Naval  Postgraduate  School  (NPS) 
students.  A  sample  size  of  nine  internal  and  external  customers  was  gathered.  They 
provided  adequate  and  generalizable  infonnation  about  the  experiences  of  AF 
Contracting’s  customers.  Although  the  sample  participants  are  locally  based,  they  bring 
experiences  from  across  the  AF. 

B.  SUBJECT  SELECTION  PROCESS 

The  subject  selection  process  involved  outreach  efforts  to  acquire  participants  for 
the  study,  and  establishing  qualification  criteria  for  potential  subjects.  Potential  subjects 
needed  to  have  experience  working  with  AF  Contracting  and  be  able  to  clearly  articulate 
their  experiences.  The  potential  subjects  also  needed  to  be  familiar  with  AF  Contracting’s 
requirement  definition,  solicitation,  source  selection,  award,  contract  management  and 
contract  close-out  processes.  The  interview  questions  used  tenninology  and  concepts  that 
require  the  participants  to  have  a  foundational  understanding  of  AF  Contracting’s 


processes. 
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Once  the  criteria  for  the  potential  subjects  were  established,  I  sought  out  qualified 
subjects.  To  assist  in  identifying  subjects,  I  requested  a  list  of  internal  and  external 
customers  from  the  contracting  office  at  Travis  AFB.  Travis  AFB’s  contracting  office 
supports  a  diverse  group  of  units,  each  with  specialized  missions.  There  are  nearly  20 
different  squadrons  on  the  base  representing  a  plethora  of  goods  and  services  sourced  by 
the  contracting  office  (“Travis  Air  Force  Base,”  n.d.).  The  contracting  office  provided  me 
with  a  list  of  their  internal  and  external  customers  who  were  qualified,  based  on  the 
previously  mentioned  criteria. 

A  second  method  for  identifying  qualified  subjects  was  through  the  Air  Force 
contingent  at  NPS.  The  Air  Force  contingent  at  NPS  consists  of  representatives  from  a 
variety  of  AF  occupations.  The  variety  of  Air  Force  students  at  NPS  provides  a  vast  pool 
of  experiences  and  interactions  with  AF  contracting  offices  across  the  world.  AF  students 
were  initially  contacted  via  a  weekly  email  distributed  by  the  local  AF  commander.  The 
email  detailed  the  qualifying  criteria  and  explained  the  purpose  and  scope  of  the  research. 

All  potential  subjects  were  initially  contacted  through  email  or  phone.  Potential 
subjects  who  were  qualified  and  interested  in  participating  in  the  research  were 
interviewed  following  the  format  of  the  interview  questionnaires  in  Appendices  A  and  B. 
The  following  section  describes  the  development  of  the  interview  questionnaires  for  both 
internal  and  external  customers. 

C.  INTERVIEW  QUESTIONNAIRE 

Before  addressing  customer  satisfaction  problems,  it  is  imperative  to  obtain  a 
baseline  understanding  of  the  customers’  perspective  about  the  organization.  For  the 
purpose  of  this  research,  this  understanding  is  accomplished  through  separate  interview 
questionnaires  for  both  internal  and  external  customers.  Each  customer  base  provided  a 
different,  yet  valuable,  perspective  on  AF  Contracting  processes  which  reinforced  the 
need  to  create  two  distinct  interview  questionnaires.  The  interview  questionnaire  served 
as  a  method  for  gathering  information  about  AF  Contracting’s  internal  and  external 
customers’  customer  satisfaction  experiences  and  expectations. 
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The  internal  customers’  interview  questionnaire  integrated  both  closed-  and  open- 
ended  questions.  The  closed-ended  questions  were  designed  to  gather  quantifiable 
information  through  the  use  of  Likert-type  questions.  The  open-ended  questions  were 
designed  to  allow  the  participants  to  discuss  their  experiences  with  AF  Contracting.  The 
internal  customer  questionnaire  also  allowed  participants  to  share  their  best  practices  as 
they  pertain  to  their  companies’  efforts  to  maintain  and  collect  customer  satisfaction  data. 

Similarly,  the  external  customers’  interview  questionnaire  was  structured  in  the 
same  manner  as  the  internal  customers’  questionnaire.  Both  open-  and  closed-ended 
questions  were  utilized  to  garner  infonnation  about  AF  Contracting  from  the  lens  of  its 
external  customers.  The  external  customers’  interview  questionnaire  used  open-ended 
questions  to  examine  AF-centric  issues,  particularly  during  the  requirement  definition 
process,  and  post-award  administration.  Requirement  definition  and  post-award 
administration  are  the  main  duties  of  external  customers  in  the  contracting  process.  Both 
the  external  and  internal  customers’  questionnaires  provided  the  participants  with  the 
flexibility  to  address  customer  satisfaction  concerns  and  expectations  of  AF  Contracting. 

D.  DATA  COLLECTION  PROCESS  AND  ANALYSIS 

The  data  collection  process  involved  telephone  and  in-person  interviews  with  four 
internal  and  five  external  customers;  however,  a  total  of  89  potential  subjects  were 
initially  contacted.  The  interviews  were  conducted  over  a  five -week  period  with  each 
interview  conversation  lasting  approximately  25  minutes.  Participants  were  provided  the 
questions  prior  to  the  interview  to  help  facilitate  the  conversation  and  allow  them  an 
opportunity  to  ponder  their  responses. 

Once  the  interviews  were  complete,  responses  to  the  closed-ended  questions  were 
summarized  according  to  the  number  of  responses  for  each  question.  Each  open-ended 
question  was  reviewed  and  sorted,  with  the  goal  of  aligning  common  trends  among  all 
responses.  The  results  section  further  defines  and  outlines  the  findings  from  the 
interviews. 
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E.  SUMMARY 

This  chapter  describes  the  methodology  employed  to  collect  and  explicate  the 
data  collection  process  for  this  research.  The  chapter  begins  with  a  discussion  about  the 
sample  size  and  selection  process  of  the  participants.  This  process  involved  identifying 
and  sourcing  potential  participants  through  multiple  avenues.  The  chapter  then  explains 
the  development  of  the  interview  questionnaire  for  AF  Contracting’s  internal  and  external 
customers.  Two  separate  questionnaires  were  created  because  the  experiences  and 
expectations  of  the  different  customer  bases  were  expected  to  yield  varied  responses,  as  it 
pertains  to  their  level  of  customer  satisfaction.  Lastly,  this  chapter  briefly  provides  and 
overview  of  the  data  analysis  process  for  assessing  and  evaluating  the  responses  from 
each  participant. 
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IV.  RESULTS 


This  chapter  details  the  results  from  the  interviews,  beginning  with  a  discussion 
about  the  participants’  demographics.  Next,  the  chapter  will  present  the  results  of  each 
question  by  describing  the  findings  and  trends  of  each.  Lastly,  the  chapter  will  compare 
and  identify  similarities  between  the  internal  and  external  customers’  interview 
responses. 

A.  PARTICIPANT  DEMOGRAPHICS 

There  were  a  total  of  five  external  customers  who  participated  in  this  research.  All 
of  the  external  customers  interviewed  had  first-hand  interaction  with  the  contracting 
process.  Most  had  experience  drafting  requirements  and  overseeing  contractor 
performance.  The  external  customers  interviewed  were  representative  of  several  AF 
functional  specialties,  each  having  unique  experiences  with  multiple  AF  Contracting 
offices.  The  participants  were  a  combination  of  military  and  civilian,  ranking  from  a 
military  equivalency  of  MSgt  (E-7)  to  Major  (0-4).  Although  the  research  was  limited  to 
five  external  customers,  their  responses  reflected  a  broad  range  of  experiences. 

There  were  a  total  of  five  internal  customers  who  participated  in  the  research.  All 
internal  customers  had  multiple  years  of  experience  supporting  AF  Contracting.  The 
internal  customers  were  fluent  in  AF  Contracting’s  processes  and  procedures  and 
provided  a  unique  perspective  on  AF  Contracting’s  customer  service  abilities.  Although 
the  research  was  limited  to  five  internal  customers,  their  responses  reflected  their 
experiences  supporting  AF  Contracting  offices  across  the  United  States.  The  internal 
customers  held  positions  within  their  companies  ranging  from  Program  Manager  (PM)  to 
Chief  Operations  Officer  (COO).  Both  the  external  and  internal  participants  shared  their 
experiences  working  with  AF  Contracting,  and  these  responses  are  detailed  in  the 
subsequent  sections  of  this  chapter. 
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B. 


EXTERNAL  CUSTOMER  RESULTS 


The  external  customers’  interview  consisted  of  nine  questions,  both  open-  and 
closed-ended.  The  interview  results  were  compiled  and  sorted  based  upon  the 
participants’  responses.  The  results  of  the  open-ended  questions  are  presented  according 
to  the  most  frequent  or  noteworthy  trends  amongst  the  participants’  responses.  The 
results  of  the  closed-ended  questions  are  presented  based  on  the  total  number  of 
responses  received.  Each  question  is  depicted  in  the  form  of  a  table  and  will  be  presented 
in  the  same  sequence  as  the  interview  was  conducted.  The  ensuing  paragraphs  report  the 
external  customers’  interview  results. 

1.  Question  #1:  On  a  scale  of  1-  5  with  1  being  “No  Value”  and  5  being 
“Very  Valuable,”  how  valuable  is  customer  satisfaction  to  your  unit? 

The  intent  of  this  question  was  to  determine  the  importance  of  customer 
satisfaction  to  AF  Contracting’s  external  customers.  The  data  presented  in  Table  1  show 
that  four  of  the  five  participants  believed  customer  satisfaction  was  “Valuable”  to  “Very 
Valuable”  to  their  representative  organizations.  During  the  interviews,  a  majority  of  the 
participants  expressed  their  support  and  appreciation  for  addressing  customer  satisfaction. 


Table  1.  Interview  Question  #1 


Question  1:  closed-ended 

1-  No  Value 

0 

2-  Little  Value 

0 

3-  Slightly  Valuable 

0 

4-  Valuable 

1 

5-  Very  Valuable 

4 

2.  Question  #2:  Do  you  currently  provide  customer  satisfaction  feedback 
to  other  service  functions?  If  so,  in  what  manner? 

This  question  was  created  to  garner  information  about  our  external  customers’ 
current  customer  satisfaction  feedback  practices.  This  question  was  also  designed  to  see 
how  they  have  been  providing  customer  satisfaction  feedback  to  other  organizations,  if 
applicable.  For  example,  our  external  customers  may  have  given  feedback  to  other  AF 
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service-oriented  functions,  like  Force  Support  Squadron  (FSS)  or  Civil  Engineering  (CE) 
squadron.  If  our  external  customers  are  providing  customer  satisfaction  feedback  to  other 
organizations,  then  there  is  an  opportunity  to  learn  how  they  are  providing  that  feedback 
(i.e.,  which  feedback  methods  they  are  familiar  with  using).  Other  service-oriented 
functions  may  be  using  customer  satisfaction  feedback  mechanisms  that  AF  Contracting 
may  benefit  from,  either  through  adoption,  or  a  derivative  of  their  current  practices.  As 
seen  in  Table  2,  all  participants  responded  that  they  had  not,  and  currently  do  not,  provide 
customer  satisfaction  feedback  to  any  other  organization.  Since  this  was  true,  no  further 
line  of  questioning  was  conducted. 


Table  2.  Interview  Question  #2 


Question  2:  open-ended 

Trend 

"no" 

5 

3.  Question  #3:  When  working  with  AF  Contracting,  what  determines 
your  level  of  satisfaction? 

The  intent  of  this  question  was  to  identify  which  characteristics  detennine  the 
external  customers’  level  of  satisfaction  with  AF  Contracting.  This  question  was  created 
to  be  open-ended,  and  allowed  the  customers  to  share  their  perspectives  about  the  factors 
leading  to  increased  levels  of  customer  satisfaction.  As  the  participants  responded  to  this 
question,  there  were  five  trends  that  emerged.  The  first  trend  was  Timeliness,  and  four  of 
the  five  respondents  agreed.  Timeliness  refers  to  how  quickly  AF  Contracting  addresses 
their  customers’  concerns.  Timeliness  also  considers  the  amount  of  time  spent  placing  an 
item  or  service  on  a  contract — the  less  time  spent  the  better. 

The  second  trend  was  Accuracy,  which  was  mentioned  by  three  of  the  five 
respondents.  According  to  the  respondents,  Accuracy  was  described  as  how  well  the 
contract  produced  the  right  product/service  requested.  Accuracy  was  viewed  as  a 
component  affecting  Timeliness,  because  if  the  wrong  or  inaccurate  item/service  was 
procured,  more  time  would  be  spent  re -procuring  the  right  item/service. 
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The  third  trend,  Communication,  was  echoed  by  two  of  the  five  respondents  and 
is  also  a  component  of  Timeliness.  As  an  item/service  is  being  procured,  the  customers 
expect  clear  lines  of  communication  throughout  the  entire  process.  When  clear  lines  of 
communication  are  established,  AF  Contracting  is  better  positioned  to  provide  a  timely 
response  to  their  customers’  concerns. 

Approachability  is  the  fourth  trend  and,  similar  to  Communication,  was 
acknowledged  by  two  of  the  five  respondents.  Approachability  addresses  interactions 
with  AF  Contracting  personnel  (vice  AF  Contracting  processes  and  procedures,  as  the 
aforementioned  trends  have  addressed).  The  participants  valued  AF  Contracting 
personnel  who  were  approachable  and  made  customers  feel  as  though  their  contracts 
were  important  and  valuable.  Approachability  also  included  being  able  to  converse  in  a 
professional  and  friendly  manner. 

Similar  to  Approachability,  Expertise  is  the  fifth  trend  and  refers  to  the 
Contracting  Officer’s  (CO)  or  Contract  Specialist’s  (CS)  technical  knowledge  of 
traditional  contracting  practices.  COs  and  CSs  are  trained  to  know  the  regulations  and 
contracting  strategies  that  are  available  to  assist  AF  organizations  in  procuring  the 
items/services  they  need.  Although,  there  was  only  one  respondent  who  mentioned 
Expertise  as  a  factor  that  determined  their  level  of  customer  satisfaction,  it  is  a 
noteworthy  response  because  a  CO  is  judicially  and  financially  responsible  for  the 
contract  actions  they  authorize.  Therefore,  it  is  imperative  that  CO’s  have  a  firm 
understanding  and  expert  knowledge  of  the  regulations  that  govern  their  trade.  Table  3 
recaps  the  results  of  this  open-ended  question: 


Table  3.  Interview  Question  #3 


Question  3:  open-ended 

Trend  #1 

Timeliness 

4 

Trend  #2 

Accuracy 

3 

Trend  #3 

Communication 

2 

Trend  #4 

Approachability 

2 

Trend  #5 

Expertise 

1 
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4.  Question  #4:  Concerning  AF  Contracting,  in  what  areas  would  you 
like  to  give  feedback? 

The  purpose  of  this  question  was  to  determine  the  aspects  of  AF  Contracting 
where  external  customers  would  like  to  provide  feedback.  The  list  in  Table  4,  found  at 
the  end  of  this  section,  was  provided  to  each  interviewee,  but  was  not  designed  to  be  an 
exhaustive  and  limited  list  of  options.  Rather,  the  intent  of  providing  the  list  was  to  assist 
the  external  customers  in  exploring  all  the  possible  areas  of  AF  Contracting.  The  last 
option  on  the  list  was  Other,  which  allowed  the  customers  to  submit  any  additional  areas 
where  they  would  like  to  provide  feedback. 

Based  on  the  participants’  responses,  there  were  three  areas  supported  by  four  of 
the  five  participants.  CO  Expertise,  CO  Timeliness,  and  CO  Communication.  These  areas 
focus  on  the  CO’s  knowledge  of  his/her  job,  as  well  as  their  ability  to  quickly  establish 
contracts,  while  maintaining  communication  with  his/her  customers.  These  findings  align 
with  the  trends  found  in  Question  #3,  and  further  emphasize  the  customers  desire  to  be 
satisfied  in  these  areas. 

There  were  three  areas  that  three  of  the  five  participants  believed  were  important 
to  give  feedback.  CO  Responsiveness,  CO  Proactivity,  and  CO  Availability.  CO 
Responsiveness  refers  to  the  CO’s  ability  to  be  approachable  and  responsive  in, 
understanding  the  customers’  needs.  This  area  is  similar  to  Approachability,  which  was 
mentioned  in  Question  #3.  CO  Proactivity  is  characterized  by  the  CO’s  foresight  and 
dedication  to  frequently  engaging  with  his/her  customers  in  order  to  mitigate  problematic 
situations.  Lastly,  CO  Availability  refers  to  their  willingness  to  be  accessible  and 
reachable  throughout  the  entire  contracting  process. 

There  were  six  remaining  areas,  each  with  two  participants  wanting  to  provide 
feedback.  The  first  area  was  CO  Innovation.  CO  Innovation  recognizes  the  creativity  and 
resourcefulness  of  the  CO  as  he/she  navigates  his/her  customers  through  the  contracting 
process.  Next,  the  Contracting  Office’s  Processes  were  another  potential  area  for 
customer  feedback.  The  Contracting  Office’s  Processes  were  divided  into  three  specific 
areas:  Solicitation,  Award  and  Administration  Processes  (encompassing  feedback  areas 

two,  three  and  four,  respectively).  The  Solicitation  Process  occurs  before  the  award  of  the 
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contract  and  often  includes  activities  such  as:  market  research,  synopsizing  the 
requirement,  and  posting  the  solicitation  for  bid.  Following  the  Solicitation  Process  is  the 
Award  Process.  During  this  process  the  solicitation  is  evaluated,  and  a  contractor  is 
selected  and  awarded  the  contract.  Once  the  Award  Process  is  complete,  the 
Administration  Process  begins.  This  process  involves  monitoring  the  contract  for 
contractor  compliance,  and  all  accompanying  administrative  actions  needed  to  ensure  the 
contract  is  completed. 

Finally,  the  fifth  and  sixth  areas  were  the  Requirements  Development  Process, 
and  Contracting  Officer  Representative  (COR)  Training.  The  Requirements  Development 
Process  occurs  before  the  solicitation  process  and  involves:  identifying  a  requirement, 
developing  a  list  of  requirement  objectives,  and  establishing  performance  standards. 
Finally,  the  sixth  area  was  COR  Training.  A  COR  is  a  formally  nominated  representative 
of  the  CO,  who  is  extended  the  authority  to  oversee  and  ensure  contractor  compliance 
with  contractual  obligations.  The  COR  is  typically  identified  during  the  requirement 
development  process  and  is  the  visual  extension  of  the  CO,  who  is  often  geographically 
separated  from  contract  performance  site. 

All  of  the  areas  mentioned  in  the  preceding  paragraphs  were  acknowledge  by  the 
customers  as  areas  they  would  like  to  provide  feedback  to  AF  Contracting.  The  customers 
were  satisfied  with  the  list  of  areas  provided  and  did  not  offer  any  additional  areas. 
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Table  4.  Interview  Question  #4 


Question  #4:  closed-ended 

Contracting  Officer 
(CO)  Expertise 

4 

CO  Responsiveness 

3 

CO  Innovation 

2 

CO  Timeliness 

4 

CO  Communication 

4 

CO  Proactivity 

3 

CO  Availability 

3 

Contracting  Office's 
Solicitation  Processes 

2 

Contracting  Office's 
Administration 

Processes 

2 

Contracting  Office's 
Award  Processes 

2 

Requirement's 
Development  Process 

2 

COR  Training 

2 

Other: 

0 

5.  Question  #5:  How  often  would  you  expect  to  give  customer 
satisfaction  feedback? 

The  intent  of  this  question  was  to  identify  how  frequently  a  customer  expected  to 

provide  AF  Contracting  with  customer  satisfaction  feedback.  For  this  question, 

participants  were  given  the  option  to  select  multiple  answers,  as  they  deemed  appropriate. 

The  five  options  for  this  question  were  Annual,  Semi-Annual,  Quarterly,  Every  Contract 

Action,  and  Other.  The  results  are  presented  in  Table  5,  and  show  that  all  participants 

believed  customer  satisfaction  feedback  should  be  collected  at  least  Quarterly. 

Additionally,  two  participants  believed  Semi-Annual  feedback  was  acceptable,  and  one 

participant  thought  feedback  after  Every  Contract  Action  was  appropriate.  There  was  one 

participant  who  suggested  an  Other  frequency  of  Monthly.  Monthly  feedback  would  be 

beneficial  depending  upon  the  requirements  and  specifications  of  the  contract  (e.g.,  a 

contract  for  ongoing  services).  All  participants  agreed  that  Annual  feedback  was  too 
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infrequent.  A  year  with  no  communication  about  the  customers’  satisfaction  was  not 
ideal. 


Table  5.  Interview  Question  #5 


Question  5:  closed-ended 

Annual 

0 

Semi-Annual 

2 

Quarterly 

5 

Every  Contract  Action 

1 

Other 

1 

6.  Question  #6:  Concerning  AF  Contracting,  what  method  of  giving 
feedback  is  preferred?  Please  rank  from  most  preferred  (1)  to  least 
preferred  (7) 

This  question  was  created  to  detennine  which  methods  the  customers  preferred  to 
use  when  giving  customer  satisfaction  feedback.  This  question  required  the  participants 
to  rank  the  methods  from  highest  to  lowest  preference.  The  list  in  Table  6  was  not 
intended  to  limit  the  customers  to  only  these  select  options.  Instead,  the  purpose  for 
creating  the  list  was  to  identify  a  variety  of  options  that  were  practical  and  able  to  be 
implemented  within  the  AF.  The  customers  were  able  to  suggest  other  methods  and  rank 
them  accordingly. 

The  first  column  of  the  table  lists  the  seven  methods  as  they  were  presented  to  the 
participants  (note:  the  seventh  method  is  “Other”).  The  third  row  from  the  top  of  the 
table,  lists  numbers  from  one  to  seven.  These  numbers  represent  a  rank  position  where  1 
was  most  preferred  and  7  was  least  preferred.  Since  the  participants  did  not  provide 
suggestions  for  a  seventh  method,  the  least  preferred  rank  position  defaulted  to  6.  This 
explains  the  absence  of  any  numbers  under  rank  position  7.  Each  rank  position  (counting 
down  the  columns)  and  each  method  (counting  across  the  rows)  totaled  five  responses. 
The  data  within  the  matrix  is  further  explained  in  the  following  paragraphs. 

Based  on  the  participants’  responses,  the  Online  Survey,  Handwritten  Survey  and 
In-person  Interview,  were  the  only  methods  that  received  a  rank  of  1 .  Two  participants 
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gave  the  Online  Survey  and  In-person  Interview,  a  rank  of  1.  One  participant  gave  the 
Handwritten  Survey  a  rank  of  1.  The  Online  Survey  would  be  a  customer  satisfaction 
survey  administered  through  a  web-based  platform.  Similar  to  the  Online  Survey,  the 
Handwritten  Survey  would  include  the  same  type  of  questions,  but  instead  customers 
would  provide  handwritten  feedback.  Lastly,  the  In-person  Interview  would  be  a  face-to- 
face  conversation  with  the  customers,  to  acquire  feedback  about  their  level  of 
satisfaction.  The  In-person  Interview  would  allow  AF  Contracting  to  dialogue  and 
pinpoint  areas  for  improvement. 

The  Online  Survey  and  Handwritten  Survey  also  received  a  rank  of  2,  each  by  one 
participant.  Telephone  Interview  was  the  only  other  method  to  receive  a  rank  of  2  (from 
three  participants).  The  Telephone  Interview  would  involve  engaging  customers  about 
their  level  of  satisfaction  over  the  phone. 

In  rank  position  3,  the  Comment  Card,  In-person  Interview,  and  Online  Interview 
were  the  preferred  methods.  Two  participants  believed  the  Comment  Card  and  the  Online 
Interview  were  preferred  in  this  rank  position,  while  one  participant  preferred  the  In- 
person  Interview.  The  Comment  Card  and  the  Online  Interview  are  the  only  methods  that 
have  not  been  previously  explained.  The  Comment  Card  is  typically  a  short  list  of 
questions  that  allow  a  customer  to  rate  an  organizations  customer  service,  while  the 
Online  Interview  utilizes  Internet  technology  to  visually  connect  AF  Contracting  with 
their  customers,  in  order  to  acquire  their  feedback. 

In  rank  position  4,  there  were  five  different  methods  preferred.  The  five  methods 
were  the  Online  Survey,  Handwritten  Survey,  Telephone  Interview,  In-person  Interview 
and  Online  Interview.  Each  participant  selected  a  different  method  for  this  position.  Next, 
rank  position  5  had  four  preferred  methods.  Two  participants  preferred  the  Handwritten 
Survey  while  the  Comment  Card,  Telephone  Interview  and  Online  Interview  received 
preference  from  one  participant  each. 

Finally,  rank  position  6,  the  least  preferred  rank,  had  four  preferred  methods.  The 
four  methods  were  Online  Survey,  Comment  Card,  In-person  Interview  and  Online 
Interview.  Two  participants  preferred  the  Comment  Card  while  the  Online  Survey,  In- 
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person  Interview  and  Online  Interview  each  had  one  participant  preference.  Table  6, 
summarizes  these  results: 


Table  6.  Interview  Question  #6 


Question  6:  closed-ended 

Rankings  (1-most  prefered  to  7-least  prefered) 

1 

2 

3 

4 

5 

6 

7 

Online  Survey 

2 

1 

1 

1 

Handwritten  Survey 

1 

1 

1 

2 

Comment  Card 

2 

1 

2 

Telephone  Interview 

3 

1 

1 

In-person  Interview 

2 

1 

1 

1 

Online  Interview 

2 

1 

1 

1 

Other 

7.  Question  #7:  In  what  ways  could  AF  Contracting  improve  customer 
satisfaction? 

The  purpose  of  this  question  was  to  identify  adjustments  AF  Contracting  could 
make  to  improve  their  customers’  satisfaction.  This  question  was  open-ended,  which 
allowed  the  customers  to  share  their  perspectives.  Based  on  the  data  obtained  from  the 
interviews,  two  trends  emerged.  The  two  trends,  in  no  particular  order,  were: 
Communication  and  Training.  Table  7,  shows  the  trends  and  the  number  of  participants 
who  supported  each  trend. 

Communication  was  a  trend  identified  by  all  five  participants,  and  encompasses 
multiple  aspects.  First,  it  refers  to  the  communication  between  the  customer  and  the  CO, 
particularly  during  post-award  contract  actions.  Post-award  contract  actions  include 
activities  such  as,  modifications,  payments,  and  other  contract  administrative  efforts.  The 
participants  desire  a  level  of  communication  that  keeps  them  abreast  of  their  contract’s 
status,  and  makes  them  feel  their  concerns  are  a  priority.  Second  Communication  can  be  a 
tool  for  mitigating  frustration,  confusion  and  tension  between  the  customer  and  the  CO. 
Third,  Communication  brings  visibility  into  the  contracting  process  and  allows  the 
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customers  to  develop  a  relationship  with  the  CO  and  CA  for  their  contract.  There  are 
instances  where  the  CO  is  geographically  separated  from  the  place  of  the  contract’s 
performance,  and  customer’s  still  need  to  know  who  to  contact  for  support.  The  visibility 
of  the  CO  is  also  very  import  because  as  military  customers  rotate  to  different  duty 
stations,  and  are  replaced  with  new  people,  the  responsibility  for  oversight  of  the 
contractor  remains.  In  this  situation,  the  CO’s  visibility  helps  the  military  customers 
recognize  who  they  should  contact  if  there  are  any  issues  or  concerns  with  the  contract. 

Training,  the  second  trend,  was  identified  by  one  participant.  Training  specifically 
addressed  the  training  of  CORs.  CORs  are  trained  in  two  phases.  The  first  phase  includes 
on-line  courses  hosted  by  Defense  Acquisition  University  (DAU).  DAU  courses  are 
designed  to  give  the  CORs  a  foundational  understanding  of  the  contracting  process  and 
their  specific  role  as  overseers  of  a  contract.  The  second  phase  of  training  teaches  the 
CORs  how  to  properly  monitor  their  particular  contract.  Since  each  contract  varies  in 
their  specifications,  contractor  requirements,  and  inspection  frequencies,  the  CORs  need 
to  be  trained  on  the  intricacies  of  their  contract.  This  level  of  understanding  is  taught  by 
the  COR’s  local  CO  or  CA  who  is  most  knowledgeable  about  the  contract.  Based  on  the 
participant’s  response,  the  training  from  the  contracting  office  did  not  sufficiently  prepare 
him/her  to  oversee  his/her  contract. 


Table  7.  Interview  Question  #7 


Question  7:  open-ended 

Trend  #1 

Communication 

4 

Trend  #2 

Training 

1 

8.  Question  #8:  Concerning  AF  Contracting,  provide  an  example  of  one 
good  and  one  bad  experience  and  explain  why  it  was  good  or  bad. 

The  purpose  of  this  question  was  to  gain  further  insight  into  what  the  customers 
experience  when  working  with  AF  Contracting.  This  question  required  the  customers  to 
reflect  on  previous  interactions  with  AF  Contracting,  and  select  one  good  and  one  bad 
experience.  Once  an  experience  was  identified,  the  customers  were  asked  to  describe  the 
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experience,  and  state  why  it  was  good  or  bad.  From  this  question  emerged  four  trends 
from  the  good  experiences  and  three  from  the  bad  experiences.  A  few  of  the  trends 
pinpoint  contracting-specific  topics  that  have  not  previously  been  discussed;  therefore, 
these  topics  will  be  explained  within  the  context  of  the  trend  they  support.  The  trends  are 
shown  in  Table  8,  in  no  particular  order. 

a.  Good  Experiences 

From  the  participants’  good  experiences,  there  were  four  trends  identified:  CO 
Knowledge,  Partnership,  Communication,  and  Contractor  Perfonnance.  CO  Knowledge 
referred  to  the  CO’s  knowledge  of  contracting  regulations,  and  applying  that  knowledge 
to  help  the  customer  acquire  the  product  or  service  needed.  There  was  one  participant 
who  identified  this  trend,  and  they  valued  the  CO’s  ability  to  work  within  the  contracting 
regulations,  while  still  addressing  contractor  problems,  providing  timely  responses  to 
inquiries,  and  offering  sound  business  advice. 

The  next  trend,  Partnership,  was  identified  by  two  participants.  Partnership  was 
viewed  as  the  feeling  of  collaboration  and  team  effort  between  the  customers  and  AF 
Contracting.  The  participants’  experiences  reflected  their  appreciation  for  a  contracting 
environment  that  promoted  partnership.  From  these  partnerships,  contracts  were 
successfully  executed  (deemed  successful  based  on  the  customer’s  standards)  and  issues 
with  the  contractor  were  quickly  and  easily  resolved.  There  was  a  particularly  noteworthy 
situation,  when  the  customer  and  the  CO  were  able  to  work  together  to  avoid  the  fallout 
of  a  protest.  According  the  Federal  Acquisition  Regulation  (FAR),  a  protest  occurs  when 
a  potential  bidder  files  a  written  complaint  about  any  of  the  following  contracting 
activities:  a  solicitation,  a  cancelled  solicitation,  a  contract  award  or  proposed  contract 
award,  or  a  termination/cancellation  of  a  contract  award  (FAR  33.101).  The  strength  of 
the  partnership  between  AF  Contracting  and  the  customer  enabled  both  parties  to  work 
efficiently,  as  they  prepared  all  the  documentation  needed  to  defend  the  AF  against  the 
protest.  In  the  end,  the  protest  was  not  upheld  and  the  participant  credits  the  partnership 
with  AF  Contracting  as  the  reason  for  success. 
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The  third  trend  was  Communication  and  it  was  identified  by  two  participants. 
Communication  has  been  a  recurring  trend  and  echoes  the  description  found  in  Question 
7.  Essentially,  the  participants  desire  insight  into  the  contracting  process.  They  would  like 
to  know  information  such  as  the  status  of  a  contract  award,  or  who  has  the  next  action  on 
the  contract.  From  the  participants’  experiences,  it  was  clear  they  recognized  the 
criticality  of  having  strong  lines  of  communication  with  their  CO. 

The  last  trend,  Contractor  Performance,  referred  to  the  exceptional  perfonnance 
of  a  contractor  in  meeting  the  criteria  and  requirements  of  the  contract.  The  participant 
who  identified  this  trend  referenced  several  occasions  where  the  contractor’s  adherence 
to  the  contract’s  specifications  provided  them  with  the  expertise  needed  to  conduct  their 
mission.  From  the  participant’s  perspective,  the  contractor’s  exceptional  perfonnance 
was  a  direct  reflection  of  a  well-written  contract  with  clearly  defined  specifications. 

b.  Bad  Experiences 

The  participants’  responses  to  this  question  also  revealed  three  trends  from  their 
bad  experiences.  The  three  trends  were  CO  Unknowledgeable,  CO  Unapproachable,  and 
Contractor  Perfonnance.  CO  Unknowledgeable,  was  the  opposite  of  the  CO  Knowledge 
trend  mentioned  earlier  in  this  section.  CO  Unknowledgeable  referred  to  the  participant’s 
experiences  with  a  CO  and/or  CA,  who  were  unfamiliar  with  standard  contracting 
practices.  According  to  the  two  participants  who  identified  this  trend,  the  CO  and/or  CA 
were  unable  to  provide  them  with  the  contracting  expertise  they  needed  to  resolve  their 
concerns  with  their  contractor’s  perfonnance. 

The  next  trend,  identified  by  one  participant,  was  CO  Unapproachable.  This  trend 
encompasses  the  CO’s  ability  to  engage  with  the  customers,  in  a  manner  that  makes  them 
feel  their  concerns  are  a  priority.  Based  on  the  participant’s  response,  the  CO  was  very 
unapproachable  and  led  the  participant  to  feel  as  though  their  concerns  were  a  burden  and 
unimportant.  This  trend  was  similarly  identified  in  Question  3,  under  the  category  of 
Approachability.  In  Question  3,  the  CO’s  approachability  was  a  factor  that  detennined 
the  participant’s  level  of  satisfaction.  Therefore,  a  CO  that  is  unapproachable  directly 
opposes  one  of  the  factors  that  determine  the  participant’s  level  of  satisfaction. 
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The  third  trend  was  Contractor  Perfonnance  and  there  were  three  participants  that 
identified  this  trend.  According  to  the  participants’  responses,  the  bad  experiences  with 
Contractor  Performance  occurred  for  two  reasons,  poorly  written  contracts  and  satisfying 
small  business  requirements.  The  participants  suggested  poorly  written  contracts  were 
one  reason  the  contractor’s  perfonnance  was  poor.  The  contracts  were  considered  poorly 
written  because  they  used  vague  language  and  confusing  word  choices  to  describe  the 
requirement. 

Traditionally,  every  contract  incorporates  a  document  that  explains  the 
specifications  and  desired  outcome  of  the  customers’  requirement.  The  contractor  then 
uses  this  document  as  instructions  for  governing  the  product  or  service  they  will  provide. 
This  document  can  be  in  the  fonn  of  a  Statement  of  Work  (SOW)  or  Perfonnance  Work 
Statement  (PWS).  Any  of  these  documents  are  susceptible  to  being  written  poorly,  which 
may  result  in  confusion  that  makes  (or  makes  it  appear  as  though)  a  contractor  perform 
poorly. 

The  second  reason  the  participants  had  bad  experiences  with  Contractor 
Performance  was  due  to  satisfying  small  business  requirements.  In  this  case,  satisfying 
small  business  requirements  led  to  the  award  of  a  contract,  to  a  contractor  who  was 
unable  to  provide  the  level  of  perfonnance  the  customers  desired.  From  the  participant’s 
perspective,  the  contract  was  awarded  to  ensure  the  contracting  office  met  their  small 
business  goals.  According  to  the  FAR,  small  businesses  should  be  given  a  fair 
opportunity  to  compete  on  government  contracts.  The  FAR  further  requires  contracting 
offices  to  establish  goals  for  increasing  the  number  of  small  business  who  participate  in 
government  contracting  (FAR  19.201(a)(b)).  This  policy  is  a  reflection  of  “acquisition- 
related  sections  of  the  Small  Business  Act  (15  U.S.C.  631,  et  seq.),  applicable  sections  of 
the  Armed  Services  Procurement  Act  (10  U.S.C.  2302,  et  seq.),  41  U.S.C.  3104;  and 
Executive  Order  121”  (FAR  19.00(a)).  Essentially,  the  contractor’s  perfonnance  did  not 
meet  the  customer’s  standards  and  additional  time  was  spent  re-procuring  the 
requirement. 
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Table  8.  Interview  Question  #8 


Question  8:  open-ended 

Good  Experience 

Bad  Experience 

Trend  #1 

CO  Knowledge 

1 

Trend  #1 

CO  Unknowledgeable 

2 

Trend  #2 

Partnership 

2 

Trend  #2 

CO  Unapproachable 

1 

Trend  #3 

Communication 

2 

Trend  #3 

Contractor  Performance 

3 

Trend  #4 

Contractor  Performance 

1 

9.  Question  #9:  Please  share  any  other  comments. 

The  intent  of  this  question  was  to  allow  the  participants  to  share  any  additional 
information  that  had  not  previously  been  mentioned  during  the  interview.  This  was  also 
an  opportunity  for  the  participants  to  share  or  expound  upon  any  aspects  of  the  interview 
that  were  not  adequately  covered.  No  additional  comments  were  received. 

C.  INTERNAL  CUSTOMER  RESULTS 

The  internal  customers’  interview  consisted  of  ten  main  questions  and  seven  sub¬ 
questions,  both  open  and  closed-ended.  The  interview  results  were  compiled  and  sorted 
based  upon  the  participants’  responses.  The  results  of  the  open-ended  questions  were 
presented  according  to  the  most  frequent  or  noteworthy  trends  amongst  the  customer 
responses.  The  results  of  the  closed-ended  questions  were  presented  based  on  the  total 
number  of  responses  received.  Each  question  was  depicted  in  the  form  of  a  table  and  was 
presented  in  the  same  sequence  the  interview  was  conducted.  The  following  paragraphs 
report  the  internal  customers’  interview  results. 

1.  Question  #1:  On  a  scale  of  1-  5  with  1  being  “No  Value”  and  5  being 
“Very  Valuable,”  how  valuable  is  customer  satisfaction  to  your  unit? 

The  intent  of  this  question  was  to  determine  the  importance  of  customer 
satisfaction  to  AF  Contracting’s  internal  customers.  This  same  question  was  also  asked  of 
the  external  customers.  The  data  shows  five  of  the  five  participants  believed  customer 
satisfaction  was  “Very  Valuable”  to  their  company.  During  the  interviews,  a  majority  of 
the  participants  recognized  customer  satisfaction  as  a  critical  component  of  their 
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company’s  success;  they  take  pride  in  making  sure  their  customers  are  satisfied.  Table  9 
displays  the  results  of  this  question: 


Table  9.  Interview  Question  #1 


Question  1:  closed-ended 

1-  No  Value 

0 

2-  Little  Value 

0 

3-  Slightly  Valuable 

0 

4-  Valuable 

0 

5-  Very  Valuable 

5 

2.  Question  #la:  A  customer  is  defined  as  “anyone  who  is  affected  by  the 
product  or  by  the  process  used  to  produce  the  product.”  Specifically, 
external  customers  are  those  who  are  “affected  by  the 
product/service”  whereas  internal  customers  are  “affected  by  the 
process  used  to  produce  the  product/service.”  Who  do  you  seek 
feedback  from?  External  customers,  Internal  customers  or  both? 

The  purpose  of  this  question  was  to  establish  a  common  definition  of  who 
qualifies  as  an  external  or  internal  customer.  Based  on  a  common  understanding  of  who 
is  an  external  or  internal  customer,  the  question  then  asks  the  participants  to  state  whether 
they  collect  feedback  from  their  external  customers,  internal  customers  or  both.  In 
addition  to  a  written  description  of  an  external  versus  internal  customer,  each  interview 
questionnaire  provided  the  participants  with  a  visual  aid,  describing  AF  Contracting’s 
external  and  internal  customers.  The  purpose  of  the  visual  aid  was  to  further  assist  the 
participants  in  clearly  understanding  the  two  different  customer  bases,  as  established  for 
this  research.  Based  on  the  participants’  responses,  all  five  participants  stated  they 
actively  request  feedback  from  both  external  and  internal  customers.  Table  10  shows  the 
participants  responses. 
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Table  10.  Interview  Question  #la 


Question  la:  open-ended 

Trend 

"Both" 

5 

3.  Question  #2:  What  method/methods  is/are  your  company  currently 
using  to  assess  customer  satisfaction? 

This  question  was  created  to  gain  insight  into  the  type  of  methods  the  participants 
were  using  within  their  organizations,  to  assess  customer  satisfaction.  Based  on  the 
customers’  responses  six  trends  emerged.  The  six  trends  were  CPARS,  Meetings,  Email, 
Customer  Satisfaction  Feedback  Tools,  Social  Media,  and  Government  Notifications.  A 
few  of  the  trends  identify  contracting  specific  topics  that  have  not  previously  been 
discussed;  therefore,  these  topics  will  be  explained  within  the  context  of  the  trend  they 
support.  Each  trend  will  be  further  explained  in  the  subsequent  paragraphs. 

The  first  trend  was  Contractor  Perfonnance  Assessment  Reporting  System 
(CPARS)  and  four  participants  use  the  outputs  of  this  system  to  assess  the  government’s 
satisfaction.  According  to  the  FAR,  CPARS  is  the  federal  government-wide  required 
system,  for  collecting  and  storing  the  past  performance  information  for  all  contractors. 
Upon  completion  of  a  contract,  or  at  least  annually,  past  performance  information  on  the 
contractor  must  be  completed.  Past  perfonnance  infonnation  inputs  are  generally 
gathered  from  either  the  CO,  CA,  COR,  program  manager,  external  customer  or  quality 
assurance,  or  a  combination  thereof  (FAR  42. 1502(a)). 

The  system  is  designed  to  provide  contractors  with  feedback  about  their 
performance  through  an  adjectival  rating  system,  and  across  metrics  that  are  detennined 
based  on  the  type  of  work  the  contractor  was  performing.  For  example,  if  a  contractor 
was  performing  a  service/information  technology/operations  type  of  work,  then  the 
contractor  would  be  evaluated  on  seven  performance  areas.  The  perfonnance  areas  are 
Quality,  Schedule,  Cost  Control,  Business  Relations,  Management,  Small  Business  and 
Other  Areas.  Each  area  receives  a  rating,  supported  by  a  narrative,  that  justifies  the  rating 
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the  contractor  was  given.  The  objective  is  to  provide  contractors  with  both  positive  and 
negative  feedback  about  their  perfonnance  (FAR  42.1503(b)(2)). 

Regardless  of  the  type  of  work  the  contractor  was  performing,  the  same  adjectival 
rating  system  must  be  used.  The  FAR  provides  detailed  definitions  for  the  following 
ratings:  Exceptional,  Very  Good,  Satisfactory,  Marginal  and  Unsatisfactory.  These  are 
the  only  ratings  that  can  be  given  to  a  contractor  (FAR  42.1503,  Table  42-1).  The 
contractors,  who  for  the  purposes  of  this  research  are  known  as  AF  Contracting’s  internal 
customers,  have  a  vested  interest  in  the  ratings  they  receive  in  CPARS,  because  this 
information  is  used  by  the  government  to  assist  in  detennining  who  will  be  awarded  a 
new  contract  (i.e.,used  in  past  perfonnance  assessments  in  subsequent  contracts). 

The  second  trend,  four  participants  used  to  assess  customer  satisfaction  was, 
Meetings.  Meetings  were  either  face-to-face  or  virtual  and  were  used  with  both  external 
and  internal  customers.  The  purpose  of  the  meetings  was  to  engage  with  the  customers, 
most  often  on  a  weekly  or  quarterly  basis,  about  their  experiences  and  any  problems  they 
were  encountering.  As  often  as  possible,  the  participants  traveled  to  the  location  of  their 
customers  to  conduct  their  meetings.  There  was  a  participant  who  mentioned  their 
company  policy  was  to  have  monthly  contact  with  their  customers.  This  contact  could  be 
accomplished  through  in-person  meetings,  telephone,  email  or  any  other  method  of 
contact. 

The  third  trend,  mentioned  by  three  participants  was,  Email.  Email  was  used  for 
both  external  and  internal  customers,  and  was  most  often  used  when  physical  meetings 
could  not  to  take  place. 

The  fourth  trend  one  participant  used  to  assess  their  customers’  satisfaction  was 
Customer  Satisfaction  Feedback  Tools.  These  tools  were  used  to  help  the  company  obtain 
confidential  and  anonymous  customer  feedback.  One  of  the  tools  mentioned  was  Dunn  & 
Bradstreet.  Dunn  &  Bradstreet  is  a  corporate  business  solutions  provider,  offering  “a 
wide  range  of  solutions  for  Risk  &  Finance,  Operations  &  Supply,  Sales  and  Marketing 
professionals”  (Dnb.com,  2015).  Dunn  &  Bradstreet  focuses  on  collecting,  analyzing,  and 
interpreting  data  for  their  companies  to  use  when  making  business  decisions  (Dnb.com, 
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2015).  Dunn  &  Bradstreet  is  one  method  the  participant  used  to  gauge  their  non-AF 
customers’  experience. 

The  second  tool  mentioned  by  the  participant  was  Survey  Monkey.  Survey 
Monkey  is  a  popular  survey  platform  with  the  capability  to  support  “Customer 
Satisfaction,  Market  Research,  Event  Planning,  Education  &  Schools,  and  Employees  & 
HR”  survey  needs  (Surveymonkey.com,  2015).  Survey  Monkey  allows  companies  to 
create  web-based  surveys  that  collect  customer  information,  and  through  extensive 
analytics,  provide  companies  with  actionable  data  (Surveymonkey.com,  2015).  The 
participant  used  Survey  Monkey  data  for  their  other,  non-AF  contracts. 

The  fifth  trend,  Social  Media,  was  used  by  one  participant  as  a  way  to  assess  their 
internal  customers’  satisfaction.  The  participant’s  company  used  social  media  platforms 
like  Facebook  and  Twitter  to  collect  information  about  their  internal  customers’ 
satisfaction.  For  this  participant,  their  internal  customers  were  also  their  employee’s.  This 
can  occur  when  a  company  is  involved  in  contracting  for  personal  services.  According  to 
the  FAR,  personal  services  are  ’’characterized  by  the  employer-employee  relationship  it 
creates  between  the  Government  and  the  contractor’s  personnel”  (FAR  37.104  (a)). 
According  to  the  Defense  Federal  Acquisition  Regulation  Supplement  (DFARS), 
examples  of  a  personal  service  are  consultants  or  healthcare  professionals  (DFARS 
237.104).  Since  the  participant’s  company  supplied  the  AF  with  personal  services,  a 
social  media  platform  was  another  way  for  them  to  obtain  customer  satisfaction 
information  through  a  less  formal  venue. 

The  sixth  and  final  trend,  mentioned  by  one  participant,  was  Government 
Notifications.  This  form  of  assessing  customer  satisfaction  pertains  particularly  to  the 
participant’s  external  customers,  who  are  part  of  the  federal  government.  According  to 
the  FAR,  federal  government  agencies  are  required  to  issues  notifications  prior  to 
terminating  a  contractor  for  poor  performance  (FAR  49.402-3).  The  two  required 
notifications  are  a  cure  and  show  cause  notice.  Before  a  notice  is  issued,  a  Letter  of 
Concern  (LOC)  may  be  used  to  forewarn  the  contractor  of  the  government’s  disapproval 
of  their  perfonnance  on  a  contract.  If  an  LOC  does  not  suffice  to  bring  the  contractor 

back  into  compliance  with  the  contract  terms,  a  cure  notice  will  be  given. 
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A  cure  notice  is  a  preliminary  notification,  issued  when  a  contractor  has 
repeatedly  failed  to  comply  with  a  provision  in  the  contract.  The  cure  notice  generally 
allows  the  contractor  ten  days  to  cure  their  failure.  In  the  event  the  contractor  does  not 
correct  the  failure,  a  show  cause  notification  is  issued.  A  show  cause  notification  requires 
the  contractor  to  show  the  government  why  their  contract  should  not  be  terminated  (FAR 
49.402-3).  Although  a  government  notification  is  not  ideal,  the  participant  understood  the 
value  of  using  the  information  in  the  notifications,  to  gauge  and  assess  the  AF’s  level  of 
satisfaction.  Table  1 1  recaps  the  results  of  this  question. 


Table  11.  Interview  Question  #2 


Question  2:  open-ended 

Trend  #1 

CPARS 

4 

Trend  #2 

Meetings 

4 

Trend  #3 

Email 

3 

Trend  #4 

Customer  Satisfaction 

Feedback  Tools 

1 

Trend  #5 

Social  Media 

1 

Trend  #6 

Government  Notifications 

1 

4.  Question  #2a:  Given  the  method  you  have  explained,  what 
measurements  do  you  use  to  assess  customer  satisfaction? 

The  intent  of  this  question  was  to  determine  what  measurements  the  participants 
use  to  assess  their  customers’  level  of  satisfaction.  This  open-ended  question  allowed  the 
participants  to  share  which  measurements,  if  any,  their  companies  use  to  determine  if 
their  customers  are  satisfied.  From  this  question,  four  trends  were  identified.  The  trends 
were  Objective,  Subjective,  Based  on  the  Contract,  and  No  Formal  Measurements.  The 
list  in  Table  12  displays  the  results. 
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Table  12.  Interview  Question  #2a 


Question  2a:  open-ended 

Trend  #1 

Objective 

2 

Trend  #2 

Subjective 

2 

Trend  #3 

Based  on  the  Contract 

2 

Trend  #4 

No  Forma!  Measurements 

2 

The  first  trend,  identified  by  two  participants,  was  Objective.  Objective  referred  to 
the  type  of  measurements  the  participants  used  to  assess  customer  satisfaction.  Based  on 
the  participants’  responses,  objective  measurements  included  factors  like,  quality  of 
product,  business  relations,  key  personnel  and  accuracy  of  invoice.  These  measurements 
allowed  the  participants  to  develop  concrete  data  about  their  customers’  experiences, 
which  are  then  used  to  develop  customer  satisfaction  metrics. 

The  second  trend,  identified  by  two  participants  was  Subjective.  Subjective 
referred  to  measurements  that  were  qualitative  in  nature  and  allowed  the  customers  to 
respond  to  a  series  of  open-ended  questions.  These  measurements  allowed  the 
participants’  customers  to  provide  narrative  information  about  their  customer  satisfaction 
experiences  while  allowing  the  participants  to  gather  detailed  and  specific  information 
from  their  customers.  The  participants,  who  identified  the  Objective  and  Subjective 
trends,  used  a  combination  of  both  measurements  to  gain  a  comprehensive  understanding 
of  their  customers’  experiences. 

The  third  trend,  Based  on  the  Contract,  was  identified  by  two  participants.  This 
set  of  measurements  was  not  derived  from  the  participants;  rather,  they  were  given  to  the 
participants  through  the  requirements  of  a  contract.  For  example,  the  participants  may 
have  a  contract  that  requires  a  monthly  report  on  specific  metrics  outlined  in  the  contract. 
As  opposed  to  creating  another  set  of  measurement  to  assess  customer  satisfaction,  these 
participants  elect  to  use  their  contract  established  metrics.  This  allows  the  participants  to 
received  performance  and  customer  satisfaction  feedback  regularly. 
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The  last  trend,  No  Formal  Measurements,  was  used  by  two  participants.  No 
Formal  Measurements  referred  to  the  participants  who  did  not  have  formal  measurements 
for  assessing  customer  satisfaction;  rather,  they  focused  on  establishing  frequent 
communication  with  their  customers.  These  participants  did  not  have  defined  customer 
satisfaction  metrics  or  a  specific  list  of  questions  for  their  customers  to  answer. 
According  to  the  participants,  constant  communication  with  the  customer  was  the  priority 
and  objective  of  this  trend. 

5.  Question  #2b:  How  does  your  company  implement  customer 
satisfaction  feedback? 

The  purpose  of  this  question  was  to  assess  how  the  participants  implement  the 
customer  satisfaction  feedback  they  receive.  This  question  was  open-ended  and 
encouraged  the  participants  to  share  the  different  techniques  and  methods  they  use,  when 
implementing  customer  satisfaction  feedback.  From  this  question  there  were  four  trends 
identified.  The  four  trends  were:  Address  Immediately,  Change  Processes,  Management 
Team  Review  and  Plan  of  Action.  The  following  paragraphs  provide  further  explanations 
about  each  trend. 

The  first  trend,  mentioned  by  three  participants  was  Address  Immediately. 
Address  Immediately,  referred  to  the  participants  taking  immediate  actions  to  correct  or 
remedy  any  negative  customer  satisfaction  feedback  they  received.  As  often  as  feasible, 
the  participants  preferred  to  use  this  mode  of  implementation,  because  it  shows  the 
participants’  customers  that  their  concerns  are  a  priority.  The  participants  believe 
customer  satisfaction  affects  future  business,  which  is  one  reason  they  strive  to  address 
customer  satisfaction  issues  immediately. 

The  second  trend  was  discussed  by  two  participants,  and  their  method  for 
implementing  customer  satisfaction  feedback  was  to  Change  Processes.  Change 
Processes  was  the  participants’  ability  to  adjust  their  company’s  processes  to  better 
accommodate  their  customers.  The  Change  Processes  method  of  implementation  allowed 
the  participants  to  improve  their  processes  to  better  align  with  their  customers’  needs.  An 
example  of  a  processes  improvement  was  sending  out  text  messages  and  emails  to 
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customers  as  reminders  when  an  event  requiring  their  action  was  approaching.  The 
reminders  were  a  result  of  the  feedback  a  participant  had  received. 

The  next  trend,  Management  Team  Review,  was  used  by  one  participant. 
Management  Team  Review  was  the  process  the  participant’s  company’s  management 
team  used  to  implement  customer  satisfaction  feedback.  The  management  team  would 
review  all  the  customer  satisfaction  feedback  and  strategically  determine  which  processes 
to  improve,  and/or  which  actions  to  take,  to  address  the  customer  satisfaction  feedback 
they  received. 

The  fourth  and  final  trend  was  Plan  of  Action.  Plan  of  Action  was  used  by  one 
participant,  and  was  the  company’s  formal  process  for  addressing  negative  customer 
satisfaction  feedback.  This  method  of  implementation  required  the  participant  to  supply 
their  customers  with  a  plan  addressing  the  customers’  concerns  and  how  the  company 
intended  to  correct  them.  Table  13  recaps  the  aforementioned  infonnation. 


Table  13.  Interview  Question  #2b 


Question  2b:  open-ended 

Trend  #1 

Address  Immediately 

3 

Trend  #2 

Change  Processes 

2 

Trend  #3 

Management  Team  Review 

1 

Trend  #4 

Plan  of  Action 

1 

6.  Question  #2c:  How  does  your  company  keep  track  of  customer 
satisfaction  once  feedback  has  been  received? 

This  question  was  created  to  gather  insight  into  how  the  participants  track  the 
customer  satisfaction  feedback  they  received.  The  participants  were  asked  to  explain  their 
procedures  and  methods  for  capturing  and  monitoring  customer  satisfaction  data.  From 
this  question,  four  trends  emerged.  The  four  trends  were  File/Database,  Program 
Reviews,  Business  Department/PM’s,  and  No  Formal  Method.  The  paragraphs  below, 
further  explain  the  four  trends. 
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The  first  trend,  mentioned  by  two  participants,  was  File/Database.  The 
participants  used  web-based  databases  to  file  and  track  their  customer  satisfaction 
feedback.  The  database  was  used  not  only  to  track,  but  also  manage,  customer 
satisfaction  infonnation.  It  was  also  a  system  that  congregated  customer  satisfaction  data 
from  multiple  streams  of  input,  and  consolidated  it  into  digestible  information.  This 
allowed  the  participants  to  quickly  ascertain  the  status  of  all  pending  actions  relating  to 
their  customers’  satisfaction.  One  participant  used  their  database  to  incorporate  their 
quality  metrics  and  CPARS  information  as  well. 

The  second  trend  was  Program  Reviews.  Two  participants  used  this  method  to 
track  their  customer  satisfaction  feedback.  Program  Reviews  were  conducted  periodically 
and  involved  reviewing  the  status  of  an  entire  program,  to  include  customer  satisfaction 
data.  During  these  reviews,  the  company’s  leadership  would  assess  the  customer 
satisfaction  feedback,  to  detennine  if  it  could  be  implemented  and  the  best  method  for 
implementation. 

The  third  trend,  mentioned  by  two  participants,  was  Business  Department/PMs. 
The  Business  Department/PMs  were  responsible  for  tracking  and  monitoring  customer 
satisfaction  feedback.  The  participants’  business  departments  would  analyze  and  prepare 
the  customer  satisfaction  information  for  review.  This  information  was  then  used  during 
company  meetings  to  look  for  ways  to  improve. 

The  fourth  and  final  trend,  No  Formal  Method,  was  used  by  two  participants.  No 
Formal  Method  refers  to  the  participants  who  did  not  have  a  fonnal,  internally  derived, 
system  established  for  tracking  customer  satisfaction  data.  Alternately,  the  participants 
would  use  the  parameters  of  a  contract  to  track  customer  satisfaction,  because  some  of 
their  contracts  required  them  to  provide  monthly  status  updates.  Another  non-formal 
method  of  tracking  customer  satisfaction  feedback  was  to  simply  use  PMs  to  interface 
with  the  customers.  The  PMs  are  responsible  for  oversight  of  an  entire  program,  and 
ensuring  the  customers  are  satisfied  is  part  of  that  responsibility.  Table  14  recaps  the 
results  of  this  question. 
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Table  14.  Interview  Question  #2c 


Question  2c;  open-ended 

Trend  #1 

File/Database 

2 

Trend  #2 

Program  Reviews 

2 

Trend  #3 

Business  Department/  PM's 

2 

Trend  #4 

No  Formal  Method 

2 

7.  Question  #3:  How  often  do  you  collect  customer  satisfaction  data? 

The  purpose  of  this  question  was  to  identify  how  frequently  the  participants 
collected  customer  satisfaction  information.  The  participants  were  asked  to  share  how 
often  they  collected  customer  satisfaction  information  from  both  their  internal  and 
external  customers.  Generally,  the  participants’  responses  to  this  question  were  associated 
with  the  methods  their  company  used  to  assess  customer  satisfaction  (reference  Question 
#2).  For  example,  for  a  participant  who  assessed  customer  satisfaction  through  CPARS, 
his/her  response  to  this  question  would  be  “annually.”  From  this  question  there  were  four 
identifiable  trends:  Quarterly,  Annually,  Monthly  and  Other. 

The  first  trend  was  Quarterly.  There  were  three  participants  who  collected 
customer  satisfaction  data  at  this  frequency.  Two  participants  collected  customer 
satisfaction  information  Annually  and  Monthly,  for  the  second  and  third  trends, 
respectively.  Lastly,  the  fourth  trend,  Other,  was  used  by  one  participant.  The  Other 
category  consisted  of  collecting  customer  satisfaction  at  frequencies  that  ranged  from 
daily  to  several  times  per  week.  Table  15  recaps  the  participants’  responses. 
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Table  15.  Interview  Question  #3 


Question  3:  open-ended 

Trend  #1 

Quarterly 

3 

Trend  #2 

Annually 

2 

Trend  #3 

Monthly 

2 

Trend  #4 

Other 

1 

8.  Question  #3a:  Is  there  a  reason  for  this  frequency? 

This  question  was  a  follow-up  to  the  previous  question,  and  sought  to  better 
understand  the  reason  the  participants  collected  customer  satisfaction  feedback,  at  the 
selected  frequency.  The  participants’  responses  varied  and  were  often  associated  with 
their  methods  for  assessing  customer  satisfaction  (reference  Question  #2).  There  were  a 
few  commonalities  amongst  the  participants:  Proactivity,  Business  Model,  Increased 
Communication,  and  Contract  Mandate.  These  trends  are  further  explained  in  the  ensuing 
paragraphs. 

Proactivity  was  the  first  trend  and  it  was  identified  by  two  participants. 
Proactivity  referred  to  the  participants’  desire  to  be  proactive  in  their  interactions  with 
their  customers.  The  participants  believed  being  proactive  and  anticipating  problems 
before  they  manifest  was  important  to  their  customers,  therefore,  the  participants  selected 
customer  satisfaction  frequencies  that  enabled  them  to  engage  with  their  customers 
regularly. 

Business  Model  was  the  second  trend  and  it  was  mentioned  by  two  participants. 
These  participants  selected  their  frequencies  based  on  their  companies’  business  models. 
Their  companies’  business  models  dictated  the  frequency  for  collecting  customer 
satisfaction  data  by  encouraging  regular  communication  with  their  customers,  and 
fostering  a  corporate  environment  that  emphasized  every  customer  concern,  no  matter 
how  small,  was  important.  In  pursuit  of  satisfying  their  organizations’  business  models, 
the  participants  aligned  the  frequency  for  collecting  customer  satisfaction  information 
with  their  organizations’  business  practices. 
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The  third  trend  was  Increased  Communication.  Increased  Communication  was 
identified  by  two  participants  and  was  the  reason  they  collected  customer  satisfaction 
data  at  the  frequency  they  selected.  The  participants  believed  the  frequency  of  collecting 
customer  satisfaction  data  facilitated  more  communication  and  interaction  with  their 
customers.  Through  the  increase  in  communication,  the  participants  became  more  aware 
and  understanding  their  customers’  needs. 

The  fourth  and  last  trend,  Contract  Mandate,  was  mentioned  by  one  participant. 
This  participant  used  the  specifications  and  mandates  within  their  contracts  to  determine 
how  frequently  they  gathered  customer  satisfaction  information.  Table  16  summarizes  the 
participants’  responses. 


Table  16.  Interview  Question  #3 a 


Question  3a:  open-ended 

Trend  #1 

Proactivity 

2 

Trend  #2 

Business  Model 

2 

Trend  #3 

Increased  Communication 

2 

Trend  #4 

Contract  Mandate 

1 

9.  Question  #4:  At  what  level  does  your  company  collect  feedback  from 
your  internal  and  external  customers? 

The  intent  of  this  question  was  to  determine  the  level  at  which  the  participants 
collected  customer  satisfaction  feedback  from  both  their  internal  and  external  customers. 
This  question  was  closed-ended  and  provided  the  participants  with  five  options.  The 
options  were  Supervisory  Level,  Manager  Level,  Customer  Level,  Per  Transaction  Level, 
and  Other. 

The  Supervisory  Level  referred  to  collecting  customer  satisfaction  feedback  from 
senior  leadership,  supervisors  and/or  upper  level  management,  while  the  Manager  Level 
was  focused  on  collecting  feedback  from  mid-tier  managers.  The  Customer  Level  looked 
at  collect  feedback  from  the  actual  customer  or  end  user  of  the  product  or  service.  The 

fourth  option  was  Per  Transaction  Level,  which  focused  on  collecting  customer 
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satisfaction  infonnation  at  the  conclusion  of  every  transaction  between  the  company  and 
their  customers.  The  Other  category  allowed  the  participants  to  submit  any  additional 
levels  they  deemed  pertinent.  The  participants’  responses  were  separated  into  Question 
#4a  and  Question  #4b  to  distinguish  between  their  internal  and  external  customers, 
respectively.  The  next  two  sections  report  the  findings  from  this  question. 

10.  Question  #4a:  If  you  collect  internal  customer  satisfaction  according 
to  question  la: 

This  question  is  a  subset  of  Question  #4,  and  first  instructed  the  participants  to 
recall  the  definition  of  an  internal  customer,  found  in  Question  #la.  Based  on  the 
participants’  response  to  Question  #la,  they  selected  (from  the  options  given  for  this 
question)  those  options  that  best  reflected  the  level  of  feedback  they  collect  from  their 
internal  customers.  All  participants’  response  to  Question  #la  was  affirmative,  in  favor  of 
collecting  internal  customer  satisfaction  feedback;  therefore,  all  participants  provided  a 
response  to  this  question. 

Four  participants  collected  customer  satisfaction  feedback  at  both  the  Supervisory 
Level  and  Manager  Level.  All  five  participants  collected  feedback  at  the  Customer  Level, 
and  three  participants  gathered  feedback  at  the  Per  Transaction  Level.  There  were  no 
participants  who  selected  Other  as  an  option.  A  majority  of  participants  selected  multiple 
options  and  commented  that  collecting  customer  satisfaction  feedback  was  easier  at 
certain  levels.  Although  the  ease  of  collecting  feedback  varied,  most  of  the  participants 
found  value  in  collecting  feedback  at  multiple  levels.  See  Table  17  for  a  summary  of  the 
participants’  responses. 
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Table  17.  Interview  Question  #4a 


Question  4a:  closed-ended 

Supervisory  Level 

4 

Manager  Level 

4 

Customer  Level 

5 

Per  Transaction  Level 

3 

Other 

0 

11.  Question  4b:  If  you  collect  external  customer  satisfaction  according  to 
question  la: 

This  question  is  a  subset  of  Question  #4,  and  first  instructed  the  participants  to 
recall  the  definition  of  an  external  customer,  found  in  Question  #la.  Based  on  the 
participants’  response  to  Question  #la,  they  selected  (from  the  options  given  for  this 
question)  those  options  that  best  reflected  the  level  of  feedback  they  collect  from  their 
external  customers.  All  participants’  response  to  Question  #la  was  affirmative,  in  favor 
of  collecting  external  customer  satisfaction  feedback;  therefore,  all  participants  provided 
a  response  to  this  question. 

There  were  three  participants  who  collected  feedback  at  the  Supervisory  Level 
while  four  participants  collected  customer  satisfaction  feedback  at  both,  the  Manager 
Level  and  Customer  Level.  Three  participants  gathered  feedback  at  the  Per  Transaction 
Level  and  there  were  no  participants  who  selected  Other  as  an  option.  A  majority  of 
participants  selected  multiple  options,  which  gave  them  a  comprehensive  understanding 
of  their  customers’  experiences.  Table  18  provides  a  summary  of  the  participants’ 
responses. 


55 


Table  18.  Interview  Question  #4b 


Question  4b:  closed-ended 

Supervisory  Level 

3 

Manager  Level 

4 

Customer  Level 

4 

PerTransaction  Level 

3 

Other 

0 

12.  Question  #5:  Thinking  about  your  experiences  when  working  with  AF 
Contracting,  what  factors  affected  your  satisfaction? 

The  purpose  of  this  question  was  to  determine  which  factors  affected  the 
participants’  satisfaction.  This  open-ended  question  was  designed  for  the  participants  to 
reflect  on  their  experiences  working  with  AF  Contracting,  and  deduce  which  factors 
affected  their  level  of  satisfaction.  The  participants  provided  multiple  responses  to  this 
question.  Their  responses  were  combined  into  four  trends.  The  trends  were  CO 
Responsiveness/Communication,  AF  Contracting  Processes,  Partnership,  and  Varied 
Contract  Preparation  Procedures.  A  few  of  the  trends  introduced  contracting  specific 
topics  that  have  not  previously  been  discussed;  these  topics  will  be  explained  within  the 
context  of  the  trend  they  support. 

CO  Responsiveness/Communication  was  the  first  trend  identified  by  four 
participants.  This  factor  was  very  important  to  the  participants  and  included  aspects  like, 
the  CO’s  ability  to  quickly  respond  to  the  participants’  inquiries,  and  the  CO’s 
availability  to  address  their  concerns.  For  one  participant,  responsiveness  also  pertained 
to  the  CORs.  The  COR’s  ability  to  quickly  respond  and  engage  with  the  participant 
affected  their  level  of  satisfaction.  CO  Responsiveness/Communication  also 
encompassed  the  CO’s  efforts  to  communicate  with  the  participants.  Communication 
expectations  of  the  CO  spanned  from  clarifications  about  the  contracts  requirements,  to 
providing  specific  and  detailed  feedback  throughout  the  entire  life  of  the  contract. 
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The  second  trend,  AF  Contracting  Processes,  was  mentioned  by  three  participants. 
AF  Contracting  Processes  was  comprised  of  four  main  aspects.  The  first  aspect  was  the 
contracts  source  selection  approach.  According  to  the  FAR,  source  selection  approaches 
vary  along  on  a  best  value  continuum,  from  Lowest  Price  Technically  Acceptable 
(LPTA)  to  Tradeoffs  (FAR  15.101).  Per  FAR  15.101-2  (a),  “The  lowest  price  technically 
acceptable  source  selection  process  is  appropriate  when  best  value  is  expected  to  result 
from  selection  of  the  technically  acceptable  proposal  with  the  lowest  evaluated  price.” 

Traditionally,  LPTA  is  used  for  non-complex  requirements,  where  technical 
capabilities  are  not  valued  more  important  than  price.  A  source  selection  approach  is 
chosen  prior  to  a  contract  being  put  out  for  bid,  and  governs  the  evaluation  process  of 
those  bids.  Upon  completion  of  the  evaluation  process,  the  apparent  winner  of  the 
contract  is  identified  and  subsequently  awarded  the  contract  (pending  any  discrepancies 
with  the  evaluation).  The  factor  affecting  the  participants  was  the  use  of  a  source 
selection  approach  that  did  not  appear  to  reflect  the  complexity  and  uniqueness  of  the 
requirement.  For  example,  a  requirement  may  be  very  complex,  require  high  knowledge 
and  specialization,  but  the  source  selection  approach  used  was  LPTA.  While  this  decision 
resides  with  the  CO  and  the  acquisition  team,  LPTA  does  not  pennit  tradeoffs,  meaning  a 
proposal  receiving  a  high  technical  rating  but  is  also  higher  in  price,  has  a  significantly 
lower  chance  of  winning  the  contract.  The  inconsistency  between  the  requirement  and  the 
source  selection  approach  adversely  affected  the  participants’  satisfaction. 

The  second  aspect  of  the  AF  Contracting  Processes  trend  was  accuracy.  Accuracy 
referred  to  the  accuracy  of  the  documents  in  a  contract.  Contracts  that  required  several 
modifications  to  correct  presumably  simple  errors  or  inaccurate  specifications  affected 
the  participants’  satisfaction.  The  third  aspect  was  execution  of  contract  actions.  This 
aspect  pertained  to  the  time  spent  waiting  for  contract  actions  to  be  executed.  AF 
Contracting  is  responsible  for  issuing  contract  actions  ranging  from  a  contract  award  to  a 
contract  modification.  Subsequently,  there  may  be  times  when  these  actions  take  more 
time  to  process  than  the  participants  expect.  The  final  aspect  of  the  AF  Contracting 
Processes  trend  was  flexibility  of  contract  metrics.  This  aspect  pertained  specifically  to 
service  contracts  that  require  contractors  to  regularly  submit  the  status  of  contract- 
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specific  metrics.  The  participants’  satisfaction  was  affected  by  this  trend  because,  they 
have  little  to  no  input  on  the  metrics  they  are  required  to  report,  and  the  metrics  are  only 
quantitative  which  limits  their  ability  to  fully  explain  their  reports. 

Partnership,  the  third  trend,  was  identified  by  three  participants.  Partnership 
referred  to  the  relationship  between  the  participants  and  AF  Contracting,  and  AF 
Contracting  and  their  external  customers.  The  participants’  satisfaction  was  positively 
affected  when  they  felt  the  relationship  between  their  company  and  AF  Contracting 
resembled  a  partnership,  where  teamwork  was  the  method  for  resolving  contract  issues. 
The  relationship  between  the  participants  and  AF  Contracting  was  very  important,  but  the 
relationship  between  AF  Contracting  and  their  external  customers  was  equally  important, 
from  the  participants’  perspective.  The  participants  took  notice  of  instances  where  the 
relationship  between  AF  Contracting  and  their  external  customers  was  rocky.  The  discord 
that  was  bred  from  those  relationships  put  the  participants  in  a  difficult  situation,  where 
they  felt  they  had  to  please  both  parties.  Based  on  the  participants’  feedback,  partnerships 
greatly  affected  their  satisfaction. 

The  fourth  and  final  trend  was  Varied  Contract  Preparation  Procedures.  This 
trend  was  identified  by  one  participant  who  took  notice  of  the  variance,  across  several  AF 
bases,  in  the  procedures  for  preparing  contractors  to  begin  working  on  a  services  contract. 
For  example,  contractors  who  will  be  working  on  an  AF  base  will  need  to  obtain  security 
credentials  before  they  can  begin  working.  At  some  bases,  the  CO  and  security  manager 
allow  the  contractors  to  start  working  while  completing  the  credentialing  process,  but 
other  bases  require  the  contractors  to  wait  until  the  entire  process  is  complete.  The 
security  processing  time  can  take  days  or  even  weeks,  which  results  in  a  contractor’s 
inability  to  provide  their  service.  The  participants’  satisfaction  was  negatively  affected  by 
the  inconsistency  of  the  procedures  for  preparing  a  contractor  to  work.  Table  19  recaps 
the  participants’  responses  to  this  interview  question. 
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Table  19.  Interview  Question  #  5 


Question  5;  open-ended 

Trend  #1 

CO  Responsiveness/Communication 

4 

Trend  #2 

AF  Contracting  Processes 

3 

Trend  #3 

Partnership 

3 

Trend  #4 

Varied  Contract  Preparation  Procedures 

1 

13.  Question  #6:  Concerning  your  experiences  with  AF  Contracting, 
please  rate  how  important  it  would  be  for  you  to  be  able  to  submit 
feedback  in  the  following  areas,  where  1  =  No  importance  and  5  = 
Very  important. 

This  closed-ended  question  was  created  to  determine  the  areas  where  participants 
would  like  to  provide  feedback  to  AF  Contracting.  The  participants  were  asked  to  reflect 
on  their  experiences  working  with  AF  Contracting,  and  rate  a  list  of  nine  possible 
feedback  areas.  The  following  rating  system  was  provided  to  the  participants:  1  equaled 
No  Importance,  2  equaled  Little  Importance,  3  equaled  Slightly  Important,  4  equaled 
Important,  and  5  equaled  Very  Important.  This  rating  system  was  applied  to  all  nine 
possible  feedback  areas.  The  nine  feedback  areas  were:  (1)  Contracting  Officer  (CO) 
Expertise,  (2)  CO  Responsiveness,  (3)  CO  Innovation,  (4)  CO  Timeliness,  (5)  CO 
Communication,  (6)  CO  Proactivity,  (7)  CO  Availability,  (8)  Contracting  Office’s 
Solicitation  Processes,  and  (9)  Contracting  Office’s  Administration  Processes.  The 
participants  were  also  given  the  option  to  suggest  Other  feedback  areas  that  were  not  on 
the  original  list.  Each  of  these  feedback  areas  was  thoroughly  defined  and  explained  in 
Question  #4  of  the  External  Customers  Results;  therefore,  this  question  will  not  delve 
into  specific  details  about  the  feedback  areas. 

Table  20  depicts  the  results  of  the  participants’  responses.  The  first  column  of  the 

table  lists  the  nine  feedback  areas  as  they  were  presented  to  the  participants  (note:  the 

tenth  area  is  “Other”).  The  third  row  lists  numbers  from  one  to  five.  These  numbers 

represent  a  rank  position  from  1  (No  Importance)  through  5  (Very  Important).  Each 

feedback  area  (counting  across  the  rows)  totals  five  responses,  and  the  numerical  values 

within  the  cells  represent  the  number  of  participants  who  gave  the  feedback  area  a 
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particular  rank.  The  data  within  the  matrix  is  further  explained  in  the  following 
paragraphs. 


Table  20.  Interview  Question  #  6 


Question  6:  closed-ended 

Rankings  (1-No  Im 

aortance  to  5-Very  Important) 

1 

2 

3 

4 

5 

Contracting  Officer 
(CO)  Expertise 

1 

1 

3 

CO  Responsiveness 

5 

CO  Innovation 

1 

1 

2 

1 

CO  Timeliness 

1 

4 

CO  Communication 

2 

3 

CO  Proactivity 

2 

3 

CO  Availability 

1 

1 

3 

Contracting  Office's 
Solicitation  Processes 

1 

4 

Contracting  Office's 
Administration 

Processes 

1 

1 

3 

Other 

1 

Based  on  the  participants’  responses,  CO  Responsiveness  was  the  only  feedback 
area  where  all  five  participants  ranked  it  5,  Very  Important.  CO  Expertise  was  given  three 
different  ranks.  One  participant  ranked  it  a  3  another  participant  ranked  it  a  4  and  the 
remaining  three  participants  gave  it  a  rank  of  5.  Next,  CO  Innovation  rankings  were 
dispersed  between  a  rank  of  2  and  a  rank  of  5.  One  participant  ranked  CO  Innovation  as  a 
2,  another  participant  ranked  it  a  3,  while  two  participants  gave  it  a  rank  of  4  and  one 
participant  ranked  it  a  5.  The  next  feedback  area  was  CO  Timeliness.  CO  Timeliness  was 
given  a  rank  of  4  by  one  participant,  and  a  rank  of  5  by  four  participants. 

CO  Communication  and  CO  Proactivity  both  received  the  same  rank  dispersion 
where  two  participants  gave  them  a  rank  of  4,  while  three  participants  ranked  them  a  5. 
The  next  feedback  area  was  CO  Availability.  CO  Availability  was  given  a  rank  of  1  by 
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one  participant,  a  rank  of  4  by  one  participant,  and  a  rank  of  5  by  three  participants.  The 
two  remaining  feedback  areas  were  focused  on  AF  Contracting’s  processes.  The 
Contracting  Office’s  Solicitation  Processes  was  given  a  rank  of  4  by  one  participant 
while  the  remaining  four  participants  ranked  it  a  5.  The  Contracting  Office’s 
Administration  Processes,  was  ranked  2  by  one  participant,  4  by  one  participant,  and  5  by 
three  participants.  Lastly,  there  was  one  participant  who  suggested  an  additional  feedback 
area.  The  suggested  feedback  area  was  COR  Capability.  The  participant  emphasized  the 
value  that  the  COR  brings  to  the  contracting  process.  The  participant  provided  multiple 
instances  where  good  COR  interaction  made  contract  administration  much  smoother  than 
a  contract  without  a  COR  or  a  disengaged  COR.  The  participant  gave  COR  Capability  a 
rank  of  5. 

14.  Question  #7:  Concerning  AF  Contracting,  what  method  of  giving 
feedback  is  preferred?  Please  rank  from  most  preferred  (1)  to  least 
preferred  (7) 

This  exact  question  was  also  asked  to  AF  Contracting’s  external  customers  (see 
Question  #6  of  the  External  Customers  Results).  The  purpose  of  this  question  was  to 
understand  the  participants’  preferences  for  giving  customer  satisfaction  feedback  to  AF 
Contracting.  The  participants  were  provided  a  list  of  six  feedback  methods.  The  six 
feedback  methods  were,  Online  Survey,  Handwritten  Survey,  Comment  Card,  Telephone 
Interview,  In-person  Interview,  and  Online  Interview.  The  list  of  methods  was  merely 
created  to  identify  a  variety  of  options  that  were  practical  and  able  to  be  implemented 
within  the  AF.  The  customers  were  able  to  suggest,  Other  methods,  and  rank  them 
accordingly.  This  question  required  the  participants  to  rank  each  method  from  highest  to 
lowest  preference.  The  following  rank  system  was  used;  the  most  preferred  method  was 
given  a  1,  while  the  least  preferred  method  was  given  a  7.  Each  feedback  method  was 
thoroughly  defined  and  explained  in  Question  #6  of  the  External  Customers  Results; 
therefore,  this  question  will  not  delve  into  specific  details  about  the  feedback  methods. 

Table  21  depicts  the  results  of  the  participants’  responses  The  first  column  of  the 
table  lists  the  six  methods,  as  they  were  presented  to  the  participants  (note:  the  seventh 
method  is  “Other”).  The  third  row  lists  numbers  from  one  to  seven.  These  numbers 
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represent  a  rank  position  where  1  was  most  preferred  and  7  was  least  preferred.  Since  the 
participants  did  not  provide  suggestions  for  a  seventh  method,  the  least  preferred  rank 
position  defaulted  to  6.  This  explains  the  absence  of  any  numbers  under  rank  position  7. 
Each  rank  position  (counting  down  the  columns)  and  each  method  (counting  across  the 
rows)  totaled  five  responses.  The  data  within  the  matrix  is  further  explained  in  the 
following  paragraphs. 


Table  2 1 .  Interview  Question  #  7 


Question  7:  closed-ended 

Rankings  (1-most  prefered  to  7-least  prefered) 

1 

2 

3 

4 

5 

6 

7 

Online  Survey 

1 

3 

1 

Handwritten  Survey 

1 

4 

Comment  Card 

1 

2 

2 

Telephone  Interview 

1 

3 

1 

In-person  Interview 

4 

1 

Online  Interview 

1 

3 

1 

Other 

Based  on  the  participants’  responses,  Telephone  Interview  and  In-person 
Interview  were  the  only  methods  to  receive  a  rank  of  1 .  In-person  Interview  impressively 
captured  four  participant  responses  while  Telephone  Interview  was  preferred  by  one 
participant.  There  were  three  methods  that  received  a  rank  of  2.  The  three  methods  were 
Handwritten  Survey,  Telephone  Interview,  and  Online  Interview.  Both  Handwritten 
Survey  and  Online  Interview  were  supported  by  one  participant,  while  Telephone 
Interview  was  preferred  by  three  participants.  Rank  position  3  included  three  methods, 
Online  Survey,  Comment  Card,  and  Online  Interview.  Both  Online  Survey  and  Comment 
card  were  preferred  by  one  participant  and  Online  Interview  was  preferred  by  three 
participants. 

Online  Survey  and  Comment  Card  received  a  rank  of  4.  Online  Survey  was 
supported  by  three  participants  and  Command  Card  was  supported  by  two  participants. 
There  were  two  methods  that  received  a  rank  of  5.  The  two  methods  were  Handwritten 
Survey  and  Online  Interview,  where  Handwritten  Survey  was  preferred  by  four 
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participants  and  Online  Interview  was  preferred  by  one  participant.  Lastly,  there  were 
four  different  methods  that  received  a  rank  of  6.  The  four  methods  were  Online  Survey, 
Commend  Card,  Telephone  interview,  and  In-person  Interview.  For  two  participants,  the 
Comment  Card  was  least  preferred,  and  Online  Survey,  Telephone  Interview,  and  In- 
person  Interview  were  each  least  preferred  by  one  participant.  The  participant  who 
ranked  In-person  Interview  as  least  preferred  explained  the  reason  for  giving  it  a  low 
ranking  was  because  they  are  geographically  separated  from  a  majority  of  their 
customers,  which  makes  In-person  Interviews  very  impractical.  If  there  was  not  a  large 
geographic  separation  from  their  customers,  then  In-person  Interviews  would  be  given  a 
higher  ranking. 


15.  Question  #8:  In  what  ways  does  your  company  believe  AF 
Contracting  could  improve  customer  satisfaction? 

This  exact  question  was  also  asked  to  AF  Contracting’s  external  customers  (see 
Question  #7  of  the  External  Customers  Results).  The  purpose  of  this  question  was  to 
identify  what  adjustments  AF  Contracting  could  make,  to  improve  their  customers’ 
satisfaction.  This  question  was  open-ended,  which  allowed  the  participants  to  freely  share 
their  perspectives.  A  majority  of  the  participants’  responses  to  this  question  were  directly 
linked  to  the  factors  that  affected  their  satisfaction  (reference  Question  #5  of  the  Internal 
Customer  Results).  The  overall  consensus  was  that;  AF  Contracting  could  improve 
customer  satisfaction  addressing  the  factors  that  affected  the  participants’  satisfaction. 
Considering  this,  there  were  four  trends  that  were  highlighted  again  in  this  question. 
These  trends  were,  Better  Feedback,  Metrics,  Communication,  and  Variance  in  Contract 
Procedures.  These  trends  have  the  same  undertones  as  the  responses  in  Question  #5,  and 
will  be  further  explained  in  the  subsequent  paragraphs. 

The  first  trend,  Better  Feedback,  was  mentioned  by  two  participants.  Better 
Feedback  referred  specifically  to  the  feedback  given  to  the  participants  during  their 
interactions  with  COs  and  CORs  while  the  contract  is  in  progress  (i.e.,  the  contract 
administration  phase).  Based  on  the  participants’  responses,  they  try  to  engage  with  the 
COs  and  CORs  to  gather  feedback  about  their  performance,  but  they  are  often  given 
general  feedback,  if  any  at  all.  The  participants  value  the  feedback  from  the  COs  and 
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CORs,  but  the  feedback  needs  to  be  more  detailed.  Thorough  feedback  allows  the 
participants  to  identify  problems  and  make  corrections  as  quickly  as  possible. 

The  second  trend  was  Metrics.  Metrics  was  mentioned  by  one  participant  and 
referred  to  the  rigidity  of  the  metrics  provided  in  the  contract.  The  participant 
acknowledged  the  need  for  metrics,  but  the  contention  surfaced  when  the  perfonnance  of 
the  contractor  is  solely  linked  to  metrics  that  do  not  provide  AF  Contracting  with  a 
holistic  view  of  the  contractors’  status.  From  the  participant’s  response,  the  metrics 
within  a  contract  are  typically  all  quantitative  measures,  but  there  are  instances  when 
qualitative  metrics  would  provide  a  clearer  understanding  of  what  is  actually  happening. 
AF  Contracting  could  improve  the  participant’s  satisfaction  by  permitting  some 
flexibility  for  the  participant  to  give  input  to,  or  provide  feedback  on,  the  metrics. 

The  third  trend,  Communication,  was  identified  by  two  participants. 
Communication  was  viewed  very  similarly  to  the  responses  found  in  Question  #5.  For  the 
participants,  communication  activities  spanned  a  full  spectrum  of  varying  expectations.  A 
few  of  these  expectations  were  returning  phone  calls  and  emails  or  sharing  the  status  of  a 
modification  to  fix  an  oversight  in  the  contract’s  PWS.  The  participants  believed 
improving  communication  would  increase  their  satisfaction. 

The  fourth  and  final  trend  was  Varied  Contract  Preparation  Procedures.  This  trend 
was  mentioned  by  one  participant,  and  expressed  their  frustration  with  the  inconsistency 
of  the  procedures  for  preparing  contractors  to  begin  working  on  a  service-type  contract. 
The  participant’s  explanation  of  this  trend  was  nearly  the  same  as  the  participants’ 
response  in  Question  #5.  The  participants  from  this  question  and  Question  #5  shared 
similar  experiences,  and  were  both  dissatisfied  with  the  process.  The  participant  believed 
that  addressing  these  inconsistencies  would  improve  his/her  company’s  satisfaction. 
Table  22  summarizes  of  the  participants’  responses. 
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Table  22.  Interview  Question  #  8 


Question  8:  open-ended 

Trend  #1 

Better  Feedback 

2 

Trend  #2 

Metrics 

1 

Trend  #3 

Communication 

2 

Trend  #4 

Varied  Contract  Preparation  Procedures 

1 

16.  Question  #9:  Concerning  AF  Contracting,  provide  an  example  of  one 
good  and  one  bad  experience  and  explain  why  it  was  good  or  bad. 

This  same  question  was  also  asked  of  AF  Contracting’s  external  customers 
(reference  Question  #8  of  the  External  Customers  Results).  The  purpose  of  this  question 
was  to  gain  further  insight  into  what  customers’  experience  when  working  with  AF 
Contracting.  This  question  requested  the  participants  to  reflect  on  previous  interactions 
with  AF  Contracting,  and  select  one  good  and  one  bad  experience.  Once  an  experience 
was  identified,  the  participants  were  asked  to  describe  the  experience,  and  state  why  it 
was  either  good  or  bad.  From  the  participants’  good  experiences  there  were  four  trends 
identified.  The  four  trends  were:  Partnership,  Communication,  CO  Responsiveness,  and 
Metrics.  From  the  participants’  bad  experiences  there  were  three  trends  identified.  The 
three  trends  were  Communication,  CO  Unresponsive,  and  Partnership.  The  trends  from 
both  the  good  and  bad  experiences  will  be  explained  in  further  detail,  in  the  paragraphs 
below. 


a.  Good  Experiences 

The  participants’  good  experiences  resulted  in  four  trends.  The  first  trend  was 
Partnership  and  was  identified  by  two  participants.  The  participants’  good  experiences 
related  to  Partnership  were  characterized  by  the  relationship  between  AF  Contracting  and 
the  participants.  Both  participants  identified  specific  situations  where  working  as  a  team 
resulted  in  successfully  meeting  the  objectives  of  the  contract  and  accomplishing  the 
mission.  One  participant’s  example  of  Partnership  took  place  shortly  after  they  were 
awarded  a  contract.  The  participant’s  company  was  geographically  separated  from  the  AF 
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base  they  were  supporting,  and  they  were  in  need  of  a  local  site  to  prepare  their  workers. 
AF  Contracting  was  accommodating,  and  allowed  the  participant’s  company  to  use  space 
in  the  contracting  office  to  prepare  their  workers.  To  the  participant,  this  signaled  AF 
Contracting’s  willingness  to  partner  and  work  together,  to  ensure  the  workers  were  ready 
to  perform  at  the  onset  of  the  contract. 

The  second  trend,  Communication,  was  mentioned  by  one  participant.  Previous 
participant  responses  to  Question  #5  and  Question  #8,  revealed  Communication 
expectations  vary  across  multiple  aspects.  In  this  particular  situation,  the  participant  was 
referencing  their  communication  with  a  COR.  The  COR  was  very  engaged  and  frequently 
communicated  with  the  participant.  The  COR  also  provided  detailed  feedback  and  was 
fair  in  her  assessment  of  the  participant’s  company’s  performance  on  their  contract.  The 
communication  between  the  COR  and  the  customer  made  the  contract  run  very  smoothly, 
from  the  participant’s  perspective. 

The  third  trend  was  CO  Responsiveness.  CO  Responsiveness  was  mentioned  by 
one  participant,  and  referred  to  the  CO’s  ability  to  be  approachable,  responsive,  and 
understanding  of  the  customers’  needs.  The  participant  had  a  very  good  experience  with  a 
CO  who  responded  quickly  to  inquiries,  and  was  easy  to  work  with. 

The  fourth  and  final  trend,  Metrics,  was  identified  by  one  participant.  Metrics, 
referred  to  the  perfonnance  measurements  in  a  contract  that  the  participant  had  been 
awarded.  The  participant  appreciated  the  metrics  because  they  provided  guidance  on  the 
performance  expectations  of  the  contract. 

b.  Bad  Experiences 

From  the  participants’  bad  experiences  three  trends  were  identified.  The  first  trend 
was  Communication.  While  Communication  has  been  referenced  multiple  times 
throughout  the  external  and  internal  customer  results,  the  two  participants  who  mentioned 
this  trend  had  bad  communication  experiences  with  their  COs.  An  example  of  a  bad 
communication  experience  for  one  of  the  participants  involved  receiving  an  LOC  and 
negative  feedback  on  CPARS  after  believing  they  had  addressed  the  CO’s  and  COR’s 
concerns.  Despite  the  participant’s  continued  efforts  to  communicate  with  the  CO  and 
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COR  throughout  the  span  of  the  contract,  the  participant  was  unable  to  get  the  feedback 
they  needed  to  ensure  they  were  making  proper  adjustments.  Both  participants  expressed 
frustration  with  the  lack  of  communication  they  experienced  when  working  with  AF 
Contracting. 

The  second  trend,  CO  Unresponsive,  was  mentioned  by  one  participant.  CO 
Unresponsive  referred  to  the  participant’s  interaction  and  communication  with  a  CO 
during  important  contract  matters.  According  to  the  participant,  the  CO  was  very  slow  in 
responding  to  the  participant’s  questions,  executing  contract  actions,  and  issuing  contract 
modifications.  For  example,  COs  are  responsible  for  issuing  a  notification  letter  telling 
the  contractor  that  the  government  intends  to  exercise  an  option  on  the  contract. 
According  to  the  FAR,  an  option  is  defined  as  “a  unilateral  right  in  a  contract  by  which, 
for  a  specified  time,  the  Government  may  elect  to  purchase  additional  supplies  or  services 
called  for  by  the  contract,  or  may  elect  to  extend  the  tenn  of  the  contract”  (FAR  2.101). 
Essentially,  the  government  has  the  right  to  extend  a  contract  for  a  specified  period  of 
time,  typically  a  year. 

Once  the  government  has  conducted  research,  and  detennined  extending  the 
contract  is  in  the  best  interest  of  the  government,  the  CO  will  modify  the  contract  to 
extend  the  tenn  of  the  contract;  this  is  called  exercising  an  option.  Prior  to  exercising  an 
option,  the  CO  will  give  the  contractor  a  preliminary  notification  of  the  government’s 
intent  to  extend  the  contract.  This  notification  is  typically  provided  to  the  contractors 
sixty  days  before  the  contract’s  expiration  (FAR  52.217-9).  According  to  the  participant, 
the  CO  did  not  issue  a  preliminary  notification,  and  was  unresponsive  to  the  participant’s 
request  for  an  update,  regarding  whether  or  not  the  contract  was  going  to  be  extended. 
This  put  the  participant  in  a  very  difficult  situation  with  his/her  employees,  because  they 
were  unable  to  let  them  know  if  they  would  have  a  job  once  the  current  contract  tenn  was 
complete. 

The  third  and  final  trend  was  Partnership.  Partnership  was  a  bad  experience  for 

one  participant,  who  recalled  multiple  situations  where  a  lack  of  teamwork  made  it 

difficult  to  execute  the  contract.  Based  on  the  participant’s  response,  when  the  contractor 

and  the  government  do  not  work  together,  it  becomes  increasingly  more  difficult  to  meet 
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the  needs  of  the  mission.  The  more  the  contractor  and  the  government  can  work  as  a 
team,  the  better  the  chances  are  of  successfully  achieving  the  contract’s  objectives.  Table 
23  summarizes  the  participants’  responses  to  this  question. 


Table  23.  Interview  Question  #  9 


Question  9:  open-ended 

Good  Experience 

Bad  Experience 

Trend  #1 

Partnership 

2 

Trend  #1 

Communication 

2 

Trend  #2 

Communication 

1 

Trend  #2 

CO  Unresponsive 

1 

Trend  #3 

CO  Responsiveness 

1 

Trend  #3 

Partnership 

1 

Trend  #4 

Metrics 

1 

17.  Question  #10:  Please  share  any  other  comments. 

The  intent  of  this  question  was  to  allow  the  participants  to  share  any  additional 
information  that  had  not  previously  been  mentioned  during  the  interview.  This  was  also 
an  opportunity  for  the  participants  to  share  or  expound  upon  any  aspects  of  the  interview 
that  were  not  adequately  covered.  Three  participants  shared  additional  comments.  These 
comments  rehashed  many  of  the  concepts  that  were  previously  mentioned  by  other 
participants.  One  participant  suggested  an  industry  day  where  the  contractors  could 
provide  inputs  on  how  AF  Contracting  could  improve  their  customer  satisfaction. 
Another  participant  was  happy  that  customer  satisfaction  was  being  addressed  and  input 
was  being  sought  from  the  contractors. 

D.  SUMMARY 

This  chapter  captured  the  results  of  the  external  and  internal  customers’ 
interviews.  The  chapter  begins  with  a  brief  description  of  the  participants’  demographics. 
The  external  customer  participants  were  a  mix  of  military  and  civilian,  while  the  internal 
customer  participants  were  each  in  leadership  positions  within  their  company.  Following 
the  description  of  the  participant  demographics,  the  external  customer  participant  results 
were  revealed.  The  external  customer  interview  questionnaire  consisted  of  nine  questions 
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all  of  which  were  explained  in  this  chapter.  Lastly,  the  participants’  responses  to  the 
internal  customer  interview  questionnaire,  consisting  of  ten  questions,  were  discussed. 
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V.  CONCLUSION 


This  chapter  concludes  this  research  by  first  discussing  some  of  the  similarities 
between  the  external  and  internal  customers’  responses.  There  were  six  questions  that 
were  asked  of  both  the  external  and  internal  customers,  for  the  purpose  of  gathering  the 
same  type  of  information,  from  different  perspectives.  Next,  the  chapter  recommends  a 
system  for  using  the  information  found  in  this  research.  Finally,  two  future  research  areas 
are  discussed. 

A.  CUSTOMER  SIMILARITIES 

The  external  and  internal  customer  satisfaction  questionnaires  were  similarly 
structured,  where  six  of  the  questions  were  created  with  the  objective  of  discovering  the 
same  type  of  information  from  two  different  perspectives.  These  six  questions  were 
applicable  to  both  the  external  and  internal  customers,  each  of  whom  provided  their 
unique  perspectives  about  the  same  customer  satisfaction  components.  Table  24  displays 
consolidated  participants’  responses. 


Table  24.  Common  Customer  Responses 


Q-E/l 

Common  Responses 

Q-l/1 

Very  Valuable 

Q-3/5 

CO  Responsiveness  &  CO  Communication 

Q-4/6 

CO  Timeliness  &  CO  Responsiveness 

Q-6/7 

In-person  Interview  &  Telephone  Interview 

Q-7/8 

Communication 

Q-8/9 

Communication  &  Partnership 

The  first  column  shows  how  the  external  and  internal  customer  interview 
questions  correspond  with  each  other.  The  Q  stands  for  question,  the  E  corresponds  to  the 
question  number  on  the  external  customer  questionnaire,  while  the  I  corresponds  to  the 
question  number  on  the  internal  customer  questionnaire.  For  example,  Q-l/1  represents 

the  first  question  on  the  external  customer  questionnaire  and  the  first  question  on  the 
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internal  customer  questionnaire.  The  right  column,  titled  Common  Responses,  identified 
the  common  responses  between  the  external  and  internal  customers  for  each  question.  For 
example,  the  Common  Response  of  Very  Valuable  is  derived  from  the  external 
customers’  responses  to  the  first  question  of  their  interview,  and  the  internal  customers’ 
responses  to  the  first  question  of  their  interview.  Each  common  response  is  further 
explained  in  the  subsequent  paragraphs. 


The  first  common  customer  response  was  Very  Valuable.  As  previously 
mentioned,  the  first  question  on  both  the  external  and  internal  customer  questionnaires 
asked  how  much  the  customers  valued  customer  satisfaction  on  a  scale  of  1  to  5,  where  1 
was  No  Value  and  5  was  Very  Valuable.  Out  of  a  total  of  ten  collective  responses,  nine 
selected  Very  Valuable.  Ninety  percent  of  the  customers  believed  customer  satisfaction 
was  very  valuable  to  their  organizations,  thus  further  emphasizing  the  need  for  AF 
Contracting  to  develop  a  mechanism  for  collecting,  evaluating,  and  strategically 
implementing  customer  satisfaction  feedback. 

The  second  common  response  identified  two  similarities  between  the  customers’ 
responses:  CO  Responsiveness  and  CO  Communication.  These  similarities  stemmed 
from  the  third  question  of  the  external  customer  questionnaire  and  the  fifth  question  of 
the  internal  customer  questionnaire,  where  the  questions  asked  the  customers  to  share 
which  factors  detennined  their  level  of  customer  satisfaction.  CO  Responsiveness  and 
CO  Communication  were  the  only  two  responses  that  were  shared  amongst  both  the 
external  and  internal  customers.  The  customers  clearly  value  a  CO  who  is  responsive  and 
communicates  well.  This  finding  is  an  opportunity  for  AF  Contracting  to  implement 
training  for  COs,  and  I  would  also  include  CORs,  on  how  to  properly  communicate  with 
their  customers.  Whether  the  mode  of  communication  is  email,  telephone  or  in-person, 
the  customers  want  to  have  a  level  of  communication  and  responsiveness  that  makes 
them  feel  they  are  a  part  of  a  team. 

The  third  common  response  identified  two  similarities  between  the  customers’ 
responses:  CO  Timeliness  and  CO  Responsiveness.  These  similarities  were  found  in  the 
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fourth  question  of  the  external  customer  questionnaire  and  the  sixth  question  of  the 
internal  customer  questionnaire,  where  the  questions  asked  the  customers  to  rate  or 
identify  the  areas  in  which  they  would  like  to  provide  feedback.  This  question  was  posed 
slightly  differently  to  each  of  the  customers.  External  customers  were  asked  to  identify 
the  areas  where  they  would  like  to  provide  feedback.  Internal  customers  were  asked  to 
rate  the  feedback  areas  on  a  scale  of  1  to  5,  where  1  was  No  Importance  and  5  was  Very 
Important.  CO  Timeliness  and  CO  Responsiveness  emerged  as  the  common  customer 
responses,  based  on  the  total  number  of  responses  received.  Since  the  internal  customers 
were  asked  to  rate  the  feedback  areas,  only  the  number  of  responses  that  were  Very 
Important  (receiving  a  5)  were  included  in  the  calculation  for  detennining  the  areas  that 
received  the  greatest  number  of  responses. 

Based  on  this  calculation  method,  both  CO  Timeliness  and  CO  Responsiveness 
were  supported  by  eight  of  the  possible  ten  customer  responses.  CO  Timeliness  and  CO 
Responsiveness  are  closely  related.  CO  Timeliness  most  often  refers  to  the  COs  quickly 
providing  the  documentation  needed  to  execute  contract  actions,  CO  Responsiveness  is 
COs  efforts  to  quickly  respond  to  the  customers’  inquiries  about  that  status  of  the  contract 
action  documents.  These  findings  show  that,  CO  Timeliness  and  CO  Responsiveness  are 
combine  to  affect  customer  satisfaction.  These  feedback  areas  are  distinctly  important  to 
the  customers,  and  AF  Contracting  should  expend  effort  to  determine  the  best  practices 
for  addressing  them. 

The  fourth  common  response  identified  two  similarities  between  the  customers’ 
feedback  mechanism  responses:  In-person  Interview  and  Telephone  Interview.  These 
similarities  were  found  in  the  sixth  question  of  the  external  customer  questionnaire  and 
the  seventh  question  of  the  internal  customer  questionnaire,  where  the  questions  asked 
the  customers  to  rank  their  most  preferred  methods  for  giving  feedback.  From  a  list  of 
seven  options  (one  option  was  Other),  the  external  and  internal  customers  ranked  the 
methods  1  through  7,  where  1  was  most  preferred  and  7  was  least  preferred.  The 
customers’  responses  were  widely  distributed  across  the  methods,  with  no  strong 
preference  towards  one  particular  method.  The  In-person  interview  was  the  only  method 
to  receive  external  and  internal  customer  responses  for  the  most  preferred  method,  while 
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the  Telephone  Interview  was  the  second  preferred  method  by  a  majority  of  the  customers. 
Although  the  customers’  responses  were  widely  distributed,  AF  Contracting  should  still 
attempt  to  engage  with  customers  in  an  environment  that  is  most  conducive  for  gathering 
customer  satisfaction  feedback.  The  environment  will  be  significantly  influenced  by  the 
level  and  frequency  with  which  the  information  is  collected.  For  example,  if  AF 
Contracting  elects  to  gather  customer  satisfaction  feedback  annually  at  the  senior 
leadership  level,  an  in-person  or  telephone  interview  is  feasible,  and  provides  depth  and 
richness  to  the  feedback. 

The  fifth  common  response,  Communication,  was  found  in  the  seventh  question 
of  the  external  customer  questionnaire  and  the  eighth  question  of  the  internal  customer 
questionnaire.  These  questions  asked  the  customers  to  explain,  ways  AF  Contracting 
could  improve  their  customer  satisfaction.  Communication,  a  trend/characteristic  that  has 
been  identified  several  times  throughout  this  research,  again  surfaced  as  common 
response  for  improving  AF  Contracting’s  customers’  satisfaction.  Communication  is  a 
broad  term  that  is  measured  differently  by  the  customers,  but  engaging  with  the 
customers  and  learning  their  communication  expectations,  will  supply  AF  Contracting 
with  the  infonnation  they  need  to  meet  the  customers’  communication  needs.  There  is 
simply  no  better  solution  for  improving  communication  than  to  communicate,  attentively 
listen,  and  strategize  the  best  method  for  addressing  the  customers’  concerns. 

The  sixth  and  final  common  response  identified  two  similarities  between  the 
customers’  responses,  Communication  and  Partnership.  These  similarities  were  found  in 
the  eighth  question  of  the  external  customer  questionnaire  and  the  ninth  question  of  the 
internal  customer  questionnaire.  The  questions  asked  the  customers  to  share  an  example 
of  one  good  and  one  bad  experience  they  encountered  while  working  with  AF 
Contracting.  While  the  customers  were  able  to  recall  situations  when  Communication  and 
Partnership  were  good,  they  also  recalled  situations  where  their  Communication  and 
Partnership  expectations  were  consistently  not  met.  Since  Communication  and 
Partnership  were  both  good  and  bad  experiences  for  the  customers,  this  may  indicate  a 
lack  of  consistency  across  AF  Contracting.  Inconsistencies  in  processes  and  lack  of 
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internal  controls  across  AF  Contracting  may  be  contributing  to  the  variances  found  in  the 
customers’  experiences. 


B.  RECOMMENDATIONS 

Based  on  the  results  of  this  research,  customer  satisfaction  is  very  valuable  to  AF 
Contracting’s  customers,  and  as  such,  attention  needs  to  be  dedicated  towards  developing 
a  customer  satisfaction  system  to  gather,  analyze  and  strategically  implement  customer 
satisfaction  feedback.  This  research  recommends  a  Six-Step  Customer  Satisfaction 
System.  The  customers’  responses  gathered  in  this  research  infonn  the  key  activities 
associated  with  each  step.  The  objective  of  the  six-step  system  is  to  incorporate  the 
findings  of  this  research  into  a  process  that  can  fully  realize  the  benefits  of  improved 
customer  satisfaction.  The  six  steps  are:  (1)  define  customer  satisfaction  objectives,  (2) 
establish  metrics  and  develop  survey  instrument,  (3)  collect/analyze  the  customer 
satisfaction  infonnation,  (4)  develop  course  of  action,  (5)  implementation  of  chosen 
course  of  action,  and  (6)  follow-up.  Each  step  is  summarized  in  Figure  5  and  is  further 
explained  in  the  following  paragraphs. 

The  first  step  is  to:  define  customer  satisfaction  objectives.  The  objective  of  this 
step  is  to  establish  the  overall  purpose  and  intent  of  the  customer  satisfaction  system. 
This  is  accomplished  through  the  completion  of  activities  such  as  determining  which 
level  and  frequency  the  feedback  will  be  collected;  defining  objectives/focus  areas  for 
detennining  organizational  success;  and  allocating  responsibility  and  accountability  as  a 
means  to  oversee  the  customer  satisfaction  process.  This  step  lays  the  foundation  for  the 
subsequent  steps  and  AF  Contracting  must  carefully  consider  the  outputs  of  this  step. 

The  second  step  is  to:  establish  metrics  and  develop  a  survey  instrument.  The 
objective  of  this  step  is  to  actually  create  the  tool  for  collecting  customer  satisfaction 
feedback.  Results  from  this  research  can  be  used  to  develop  questions  pertinent  to  the 
areas  where  customers  would  like  to  provide  feedback.  For  example,  a  few  of  the  areas 
identified  by  the  customers  were  CO  Responsiveness,  CO  Timeliness  and 
Communication.  From  these  areas,  a  potential  question  might  be,  “On  a  scale  of,  1  to  5 
with  1  being  “Not  Good”  and  5  being  “Very  Good,”  how  responsive  has  AF  Contracting 
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been  at  addressing  your  concerns  and  resolving  your  contracting  problems?”  Along  with 
developing  questions,  this  step  also  includes  activities  such  as:  detennining  the  metrics  to 
measure  the  objectives  established  in  Step  1;  linking  the  questions  to  the  metrics;  and 
inputting  the  question  into  a  survey  instrument. 

Step  3  is  to:  collect/analyze  the  customer  satisfaction  infonnation.  The  purpose  of 
this  step  is  to  gather  the  feedback  from  the  customer  satisfaction  mechanism  created  in 
Step  2  and  search  for  trends  or  exceptional  responses  from  the  customers.  Once  the  trends 
and  exceptional  responses  have  been  identified,  Step  4  seeks  to  strategize  the  best  options 
for  addressing  these  responses.  Step  4  involves  developing  courses  of  action  (coa),  and  a 
few  of  the  key  activities  include  considering  policy,  process  or  personnel  adjustments; 
seeking  best  practices;  and  determining  the  best  course  of  action. 

Implementation  of  chosen  course  of  action  is  the  fifth  step  and  the  objective  is  to 
decide  how  the  course  of  action  identified  in  Step  4  should  be  implemented.  This  step 
includes  activities  such  as  detennining  if  the  course  of  action  will  be  phased  into  the 
contracting  process,  or  if  an  immediate  change  is  more  appropriate,  implementing  the 
course  of  action  at  the  appropriate  level. 

The  sixth  and  final  step  is:  follow-up.  During  follow-up  a  few  key  activities  take 
place.  These  activities  include  periodic  review  of  the  implemented  customer  satisfaction 
feedback;  reviewing  the  metrics  for  adjustments;  and  course  correct  any  customer 
satisfaction  system  processes  as  necessary.  Figure  5  summarizes  the  Six-Step  Customer 
Satisfaction  System. 
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Figure  5.  Six-Step  Customer  Satisfaction  System 


Step  1 


Define  Customer  Satisfaction 
Objectives 


•Define  objectives/focus  areas  for  organization  success 
•Determine  the  level  customer  satisfaction  will  be  collected 
•Determine  frequency  for  collecting  feedback 
•Allocate  responsibility  for  accountability  and  oversight 


Step  2 


Establish  Metrics  &  Develop 
Survey  Instrument 


Step  3 


Collect/Analyze  the  Customer 
Satisfaction  Information 


■Determine  metrics  required  to  measure  objectives 

•Develop  question  bank 

•Link  questions  to  metrics/objectives 

•Place  questions  in  survey  instrument  (e.g.,  online,  papercopies, 
smartphone  app,  etc.) 


•Implement  the  survey  instrument 
•Sort  and  analyze  the  data 
•Identify  trends 


Step  4 


Develop  Course  of  Action 


•Consider  policy,  process  or  personnel  adjustments 
•  Seek  best  practices 
•Determine  best  course  of  action 


•Consider  phased  vs  immediate  changes 
•Implement  at  appropriate  level 


•Periodic  review  of  implemented  feedback 
•Review  metrics  measurements 
•Course  correct  as  necessary 


C.  FUTURE  RESEARCH  AREAS 

This  research  is  primarily  focused  on  gathering  customer  satisfaction  feedback  to 
assist  in  identifying  pertinent  customer  satisfaction  elements  (i.e.,  customer  satisfaction 
focus  area  for  AF  Contracting).  Based  on  the  customers’  responses,  this  paper 
recommends  a  Six-Step  Customer  Satisfaction  System  for  gathering,  analyzing  and 
strategically  implementing  customer  satisfaction  feedback.  Naturally,  all  the  outputs  and 
specific  details  of  the  system  are  beyond  the  scope  of  this  research.  Delving  deeper  into 
the  key  activities  and  defining  the  outputs  of  each  step  is  a  future  area  to  research.  A 
second  and  final  future  research  area  to  consider  is  using  the  data  gathered  in  this  study  to 
create  a  customer  satisfaction  feedback  tool.  This  research  could  also  include  pilot  testing 
the  tool  and  suggesting  methods  for  implementing  the  feedback  obtained. 
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APPENDIX  A:  EXTERNAL  CUSTOMER  QUESTIONNAIRE 


External  Customer  Question  #1 

1)  On  a  scale  of  1-  5  with  1  being  “No  value”  and  5  being  “Very  valuable,” 
how  valuable  is  customer  satisfaction  to  your  unit? 


(1)  No  value  (2)  Little  value  (3)  Slightly  valuable  (4)  Valuable  (5)  Very 
valuable 


2)  Do  you  currently  provide  customer  satisfaction  feedback  to  other  service 
functions?  If  so,  in  what  manner? 


Customer  Response: 


2a)  Do  you  currently  collect  customer  satisfaction  feedback?  If 
you  do  collect  customer  satisfaction  feedback,  how? 


Customer  Response: 


3)  When  working  with  AF  Contracting,  what  determines  your  level  of 
satisfaction? 


4)  Concerning  AF  Contracting,  in  what  areas  would  you  like  to  give 
feedback? 


Contracting  Officer  (CO)  Expertise 

CO  Responsiveness 

CO  Innovation 

CO  Timeliness 

CO  Communication-the  underlying  issue 

CO  Proactivity 

CO  Availability 

Contracting  Office’s  solicitation  processes 

Contracting  Office’s  administration  processes 

Contracting  Office’s  award  processes 

Requirement’s  development  process 

COR  training 

Other: 
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5)  How  often  would  you  expect  to  give  customer  satisfaction  feedback? 


Annually 

Semi-annually 

Quarterly 

Every  contract  action 

Other: 

6)  Concerning  AF  Contracting,  what  method  of  giving  feedback  is  preferred? 
Please  rank  from  most  preferred  (1)  to  least  preferred  (7). 


Online  survey 

Handwritten  survey 

Comment  card 

Telephone  interview 

In-person  interview 

Online  interview 

Other: 

7)  In  what  ways  could  AF  Contracting  improve  customer  satisfaction? 


8)  Concerning  AF  Contracting,  provide  an  example  of  one  good  and  one  bad 
experience  and  explain  why  it  was  good  or  bad. 


Good  Experience: 
Bad  Experience: 


9)  Please  share  any  other  comments. 


Customer  Response: 


80 


Air  Force  Contracting  Customers 


Subcontractor 

#1 


Subcontractor 

#2 


Subcontractor 

#3 


Internal  Customers 


Prime 

Contractor 

#1 


Prime 

Contractor 

#3 


Prime 

Contractor 

#2 


External  Customers 


Functional 

#1 


Functional 

#3 


\ 


Functional 

#2 


The 

Warfighter 


/ 
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APPENDIX  B:  INTERNAL  CUSTOMER  QUESTIONNAIRE 


Internal  Customer  Interview  Questions 

1)  On  a  scale  of  1-  5  with  1  being  “No  value”  and  5  being  “Very  valuable,” 
how  valuable  is  customer  satisfaction  to  your  company? 


(1)  No  value  (2)  Little  value  (3)  Slightly  valuable  (4)  Valuable  (5)  Very  valuable 


la)  A  customer  is  defined  as  “anyone  who  is  affected  by  the 
product  or  by  the  process  used  to  produce  the  product.” 
Specifically,  external  customers  are  those  who  are  “affected  by 
the  product/service”  whereas  internal  customers  are  “affected 
by  the  process  used  to  produce  the  product/service.”  Who  do 
you  seek  feedback  from?  External  customers,  Internal 
customers  or  both? 

2)  What  method/methods  is/are  your  company  currently  using  to  assess 

customer  satisfaction? 


Customer  Response: 


2a)  Given  the  method  you  have  explained,  what  measurements 
do  you  use  to  assess  customer  satisfaction? 


Customer  Response: 


2b)  How  does  your  company  implement  customer  satisfaction 
feedback? 


Customer  Response: 


2c)  How  does  your  company  keep  track  of  customer 
satisfaction  once  feedback  has  been  received? 


Customer  Response: 


3)  How  often  do  you  collect  customer  satisfaction  data? 


Customer  Response: 


3a)  Is  there  a  reason  for  this  frequency? 


Customer  Response: 
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4)  At  what  level  does  your  company  collect  feedback  from  your  internal  and 
external  customers? 

4a)  If  you  collect  internal  customer  satisfaction  according  to 
question  la: _ 


Supervisory  level 

Manager  level 

Customer  level 

Per  transaction  level 

Other: 

4b)  If  you  collect  external  customer  satisfaction  according  to 
question  hi: 


Supervisory  level 

Manager  level 

Customer  level 

Per  transaction  level 

Other: 

5)  Thinking  about  your  experiences  when  working  with  AF  Contracting, 
what  factors  affected  your  satisfaction? 


Customer  Response: 


6)  Concerning  your  experiences  with  AF  Contracting,  please  rate  how 
important  it  would  be  for  you  to  be  able  to  submit  feedback  in  the  following 
areas,  where  1  =  No  importance  and  5  =  Very  important. 


(1)  No  importance  (2)  Little  importance  (3)  Slightly  important  (4)  Important  (5)  Very 

important 


Contracting  Officer  (CO)  Expertise 

CO  Responsiveness 

CO  Innovation 

CO  Timeliness 

CO  Communication 

CO  Proactivity 

CO  Availability 

Contracting  Office’s  solicitation  processes 

Contracting  Office’s  administration  processes 

Other: 
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7)  Concerning  AF  Contracting,  what  method  of  giving  feedback  is  preferred? 
Please  rank  from  most  preferred  (1)  to  least  preferred  (7). 


Online  survey 

Handwritten  survey 

Comment  card 

Telephone  interview 

In-person  interview 

Online  interview 

Other: 

8)  In  what  ways  does  your  company  believe  AF  Contracting  could  improve 
customer  satisfaction? 


Customer  Response: 


9)  Concerning  AF  Contracting,  provide  an  example  of  one  good  and  one  bad 
experience  and  explain  why  it  was  good  or  bad. 


Good  Experience: 
Bad  Experience: 


10)  Please  share  any  other  comments. 


Customer  Response: 


AF  Contracting  Customers 
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